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INTEGRATED SYSTEM FOR 
IMAGING AND MODELING THREE-DIMENSIONAL 
OBJECTS 

Technical Fidd 

The piesent invention relates generally to systems that document die 
geometry and aOxr attributes of objects in three dimensions and, specifically, to 
a system that enqjloys a scanning lidar (range finding laser) to quickly and 
accurately sense the position in duee-dimensional space of selected points on 
the sinftee of an object to generate a point cloud which represents the sensed 
positions of the selected points; that recognizes geometric ^pes represented by 
groups of points in the point cloud, and that generates a model that represents 
these geometric shapes. The model may be transformed into a further model 
usable by computer-aided design (CAD) tools, including conventional CAD 
tools. 

Background 

Mining the geometry (shape, dimensions and location) and otho- 
attributes (e.g., color, texture and reflectance imensity) of complex real obje<^: 
(whether small components such as small mechanical parts or large objects sudi 
as buildings and sites) has conventionally been a tedious and time consuming ^ 
process. That is, such measurement have traditionally been performed 
manually. In addition, transforming diese measurements into drawings or 
computer modek tequhed manual drafting or input into a CAD system for the 
production of the drawing or computer models. 

Recently innovations have endeavored to simplify this process, but all 
have fallen short of achieving full integration, automation, precision, speed and 
range. For example, in the building industry, mapping a structure 
conventionally requires three basic stq>s: 

1 . Field data gathering 

2. Data reduction and preparation 

3. Drafting and CAD 



The field data gathering step is perfoimed by a team of surveyors who manually 
measure and record dimensions of pertinent components of the structure such as 
walls, ceilings, beams, columns, doors, windows, fixtures, pipes, conduits and 
equipment. The surveyors attempt to determine the geometry of the 
components as well as the relative location of the components in the structure. 
The surveyors recorded the data in a field notebook. The field-collected data is 
then organized and reduced to tables and organized sketches, and a CAD 
operator or drafter utilizes these tables to generate final drawings or models. 

This process is labor intensive, time consuming, and error prone. In 
addition, using traditional surveying methods, the number of points which can 
actually be measured is very limited, due to the high cost of acquiring each 
point in terms of time and effort. Furthermore, if it is desired to acquire color, 
texture and other attribute information, additional field notes must be taken 
(e.g., still photographs and video). 

Recently, the field step has been somew^t automated by using a laser 
ranging device built into or mounted on an elecUx>nic theodolite. Precision 
reflection targets (retro reflectors) are placed at the locations of the object for 
which measurements are desired. Then, the laser ranging device obtains a 
precise measurement of the distance b^ween the instnuncnt and the target, 
which the theodolite provides an accurate indication of the horizontal and 
vertical angle offsets to the point relative to a given coordinate system. The 
distance and angle data are either recorded automatically on a magnetic device 
connected to the instrument or are reduced within the instrument to Cartesian 
coordinates relative to the instrument axes. This procedure is then repeated as 
many times as necessary to map a desired number of points of the object. The 
collected coordinates data can then be plotted directly on a CAD system. 

Unfortunately, the plot is of little practical use since it does not indicate 
the object geometry. Moreover, because of the requirement for retro reflectors 
which must be manually placed, and because of the relatively long time per 
reading required by the laser range finder, the gathering of sufBcient points to 
describe most objects is very labor intensive, time consuming and error prone. 
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Another known field gathering data process employs stereo pholcq g t i |phy 
and aerial idiotogfaninietry. That is, stereoscopic images are taken of ^ 
objects and the restdtii^ stereo pliotognv)hs are registered either manually cmt 
using computerized techniques to reproduce tiie relative location of the canum 
picture plane location at the time each photograph was taken. The data 
reduction and preparation step is pofonned manually by a specially trained 
opertUor. Specifically, with the aid of specially mounted stereoscopic viewing 
lenses, the operator digitizes the coordinates of a sufficient number of points to 
allow the definition of the objects using the stereo photographs. Again, the 
digitized data is input into a CAD system w is manually drawn on piqier. 

Snmmaiy 

Tlw present invention is an integrated system for generating a mbddl W a 
tiuee-dimensional olgect A scamiing laser device scans the tiiree-dinMnsiobil 
object and generates a point cloud. The points of the point cloud each indiciAe a 
location of a corresponding point on a surface of the object A first model is 
generated, responsive to the point cloud, representing consUtuent geometric 
shapes of the object. A data file is generated, responsive to the first model, 
can be inputted to a computer-aided design system. 

The subject invention fiirdier includes a method of controlling the timing 
of ou^Hit pulses from a laser for use in a device which requires scaiming of the 
laser ou^Mit, wherein each output pulse is generated in response to a pump pulse 
conqnisiQg Ac stq>s of: monitoring the time delay between die initiation of the 
pump pulses and Ibc subsequent generation of the associated output pulsei^ 
piedietiiig the time delay between the initiation of next pump pulse and the 
associated output pulse based on the monitored time delays and; initiating the 
next pump pulse at a time selected to insure the output pulse is generated it i 
time to permit proper positioning of the laser output during the scan of the 
beam. 

The present invention fivtiier includes a method of manually separating 
from a plurality of clouds of points, representing tiiree-dimensional features in a 
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scene, a subset of the points that icpresents a desired feature in the scene, the 
method comprising: selecting all the point clouds that include at least some 
data points rqnesenting the desired feature; and changing a view of the clouds 
and drawing a polygonal lasso to refute a selected subset of points to be 
5 included in a point sub-cloud and repeating the refining as many times as 

required to obtain the desired sub-cloud. 

The present invention further includes a method for automatically 
segmenting a scan field of a scene into subsets of points that represent difTerent 
surfaces in the scene, comprising the steps of: separating the scan field into a 

1 0 depth grid that includes depth information for scanned points of surfaces in the 

scene and a normal grid that includes an estimate of a normal to scanned points 
of the surfaces: convolving the depth information of the depth grid to generate a 
depth rating image whose values represent a gradient of depth change from one 
scanned point to another scaiuied point in the scene; convolving the components 

15 of the normal grid to generate a scalar value for each component for each point 

of the normal grid; for each point of the normal grid, determining from the 
scalar values for the components of that particular point a gradient of the normal 
at that point, wherein the gradients determined for the points of the normal grid 
collectively constitute a normal rating image; converting the depth rating image 

20 to a binary depth image using a recursive thresholding technique; converting the 

normal rating image to a binary normal image using a recursive thresholding 
technique; combining the binary depth image and the binary normal image to 
determine a single edge image; and grouping subsets of non-edge points as 
belonging to corresponding surfaces of the scene. 

25 The method can further include the steps of determining the type of 

geometric primitive that would best first each group of points; fitting the 
geometric primitive to the data points; and intersecting adjacent planar regions 
in the scene. 

The subject matter further includes a method for fitting a point cloud 
30 representing a comer, comprising: determining a fit of three planes to the points 

of the point cloud and creating the planes for a model; determining the three 



lines at the intersection of pairs of planes and creating the lines for the 

and detennining the veitex point at the intersection of the three planes and !~ " 

creating a vertex point for the model. 

The subject inventicm finther includes a method for modeling a three- 
dimensional scene, comprising: generating a plurality of points that each 
represent a point on a surface of the scene; detemiining a best fit of a cylinder 
for a group of the points using surftce nonmal estimates and global enor 
minimization. 

The subject invention fiirther includes a method for modeling a thite- 
dunensional scene, comprismg: generatii^ a plurality of points that each 
repreaou a point on a sur&ce of the scen^ determining a best fit of a cylindtt 
for a groiq) of the points using a quadric surftce fit and global error 
mmnfH Tft rtiofia 

The sulQect invention finther includes a method for modeling a dwee- ^ 
dimensional scene, ccmqmstng: generating a plurality ofpoints that each 
represent a point on a surface of the scene; detennining a best fit of a sphere for 
a giotq» of flie points using a quadtic surface fit and global enor muumization. 

The Sttbgect invention fiu:tiier includes a method for modeling a thne- 
dimenaonal scene, comprising: generating a plurality of points that each 
represent a point on a surface of the scene; determining a best fit quadric 8UijiK»e 
for a group ofpoints; and determining «4iich geometric primitive of a plurality 
of the family described by the quadric surface best fits the group ofpoints. 

The subject invention fiirther includes a method for merging two 
geometric primitives of tiie same type to fonn a single geometric primitive 
tiw type, craqnising: creating a new groiq) ofpoints by combining the ptrints 
used to originally fit each of the two primitives; and fitting tiie new geometric 
primitive using any appropriate fitting technique and the newly generated poliit 
gmup with points fiom each of the original primitives. 
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The subject invention further includes a method of registering a first 
model, consisting of a plurality of points and geometric |ximitives and having a 
first coordinate system, with a second model, consisting of a plurality of points 
and geometric primitives and having a second coordinate system, comprising: 
5 identifying by a user common features of the first and second scenes; 

identifying a transformation between coordinate systems that is responsive to 
the identification; and transforming the objects of the second model so that they 
use the first coordinate system. 

The subject invention further includes a method of warping, comprising: 

1 0 selecting one or more models represented by a plurality of point clouds and 

geometric primitives; specifying constraints on the locations of any number of 
points or geometric primitives; creating an artificial volume that suirounds the 
points and geometric primitives in each view and assigning mechanical material 
characteristics to the surrounding volume; computing a minimum energy 

1 5 configuration for the material in the surrounding volume in which the points or 

geometric primitives are embedded such that the configuration satisfies all 
applied constraints; and displacing the points and primitives in accordaitce with 
the computed minimum energy configuration of the surrounding volimie of 
material. In die latter metfiod, the constraints can be specified to eliminate 
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The subject invention further includes an integrated system for 7 ' 
generating a model of a dueeHiimensional scene, comprising: a scanning laser 
device that scans the three dimensional scene with pulsed laser beam wha%tn 
the pulses of light last less than 1 nanosecond with up to ^ in each pulse and 
measures the time delay, with a resolution of 30 psec or less, between each 
emitted pulse and a coiresponding pulse that is returned irom a surface of the 
scene and wherein said scanning laser device ftirther tracks and measures the 
angular orientation of the beam during the scan; and means for generating a 
point cloud based upon tfie measured time delays and angle measurements, fte 
point cloud comprising a plurality of data points that each represents a locattogn 
of a corresponding point on the sutftce. 

The subject invention further includes a system for calibrating die 
measuring dectranics in a device yMch requires monitoring tiie time of fiig^ 
of the output pulses from a laser comprising: a single mode optical fiber wldt 
one end thereof positioned to receive the output pulses of the laser, said single 
mode optical fiber having a known length; a detector positioned at one of the 
ends of the fiber fisr monitoring v/ben the pulses exit the fiber and generating a 
signal in response thereto, said signal being passed through the measuring 
electronics; and a processor for calculating a theoretical length of the fiber 
based on the detection of the pulse exiting the fiber and comparing that 
calculated length with known length of the fiber to calibrate the measuring 
electronics. 

The optical fiber can include partial reflectors located at each end thereof 
so that for each laser pulse entering the fiber a train of pulses will exit the fiber 
and wherein said train of pulses are used to finther calibrate the measuring 
electronics. 

The qrstem can fiirtiier include delay measurement electronics and 
wherein the train of pulses have a fixed delay therebetween whereby the 
monitoring of the train of pulses can be used to calibrate the delay electronics. 

The system can further include ameans for wying the power of the 
pulses monitored by the detector and wherein said detector fimctions to generate 
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a signal when the power of the detected light exceeds a predetermined threshold 
and wherein said processor functions to track the variation in the delay of the 
generation of the output signal by the detector as a function of the power of the 
output pulses, said processor further ftinctioning to calibrate the measurement of 
5 the delay based on the measured power of successive pulses used for monitoring 

the time of flight. 

The subject invention further includes an apparatus for obtaining 
position information about surface points of a three dimensional object 
comprising: a laser for generating an ou^ut beam; scanning system for moving 

1 0 the laser beam over the object; monitoring system for automatically measuring 

the range to the object based on the measurement of the reflection of the laser 
beam, said monitor system also tracking and measuring the angular position of 
the laser beam, said monitoring system having a positional accuracy for each 
point in three dimensional space equal to or better than six millimeters at one 

1 5 standard deviation over a range of up to 100 meters. 

Each range measurement can be made in under 0.005 seconds. The laser 
can generate a pulsed output and the enorgy per pulse can be less than 0.2 micro 
joules and the average output power of the laser can be less than 1 .0 milliwatts. 
The subject invention further includes an apparatus for measuring the 

2 0 distance to an object comprising: a laser for generating a beam of output pulses; 

a monitoring system for measuring the distance to the object based on the 
reflection of the laser beam, said monitoring system having an accuracy equal to 
or better than 6 millimeters at one standard deviation over its entire range of up 
to 1 00 meters and wherein each measurement can be made in less than 0.005 

2 5 seconds and wherein the laser has an energy per pulse of no more than 0.2 micro 

joules and an average power of no more than 1 milliwatt. If the object is 
provided with retro reflectors and where the range of operation is up to one 
mile. 

The subject invention further includes an apparatus for acquiring three 
30 dimensional information from a remote object c(Knprising: a scanning laser 

module for measuring position information of the object; a video module for 
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captxaing im^e infoimation from the object; and a processor for rendering a 
model of the object which includes the position infonnation and the image 
information. 

5 The video image information can be collected in a spatially coincidoadt 

manner with the measurement of position information. The video image 
information can be collected fixnn points adjacent to the points where positicp 
infonnation is obtained. 

The subject invention further includes an apparatus for obtaining 

1 0 positional information about surfiKe points of a three dimensional object 

comprising: a scanning module for measuring diree dimensional position 
inftMmation about an object; a video module for ciqrtuiuig and displaying ima^s 
information from the object; and a processor operating with the scanning awl 
video nKxhdes and permitting the use of the image information captured by 3^ 

15 video module to aid in targeting the scanning module. The processor can M 

function to specify a portion of the olgect to be targeted by the icrjititiiwg moilHte 
by dragging die image of an outline over the video image of the area to be 
targeted. 

The subject invention iiitther includes an iqjparatus for obtaining 
2 0 positional information about surface points of a three dimensional object 

comprising: a scanning module for measuring three dimensional position 
infonnation about an object; a video module for displaying image information 
obtained from the scanning module; a processor opoating with the scanning And 
video modules and permitting the use of the image infoimation displayed by 
25 said video nKxble to further refine the targeting ofthe scanning module. 

The subject invention further includes an apparatus for obtaining 
positional infonnation about sur&oe points of a three dimensional object 
conqvising: a scanning module for measuring three dimensional portion 
information about an object, said scanning module including a laser for emitting 
30 a beam of visible radiation; and a processor for controlling the scanning module 

and wherein said laser can be manually positioned so that the visible beam will 
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target the portion of the object to be scanned in response to a control signal from 
the processor. 

The subject invention further includes a system for calibrating the 
measuring electronics in a device which requires monitoring frequency changes 
5 in a light beam generated by a laser used to measure distance to an object, 

wherein said beam has frequency chirp imposed thereon comprising a single 
mode optical fiber with one end thereof positioned to receive light from the 
laser; a detector positioned to receive light which has traveled through and 
exited the fiber in combination which light firom the laser which has not traveled 

1 0 through the fiber, said detector for monitoring the changes in the frequency of 

the combined beam; and processor for determining the linearity of the chirp on 
the beam based on uniformity of the frequency changes measured by the 
detector and using the result to calibrate the measuring electronics. 

The fiber can have a known length and includes a partial reflector on 

1 5 said one end and at least a partial reflector on the other end, and wherein light 

reflected from said one end of the fiber which has not traveled in the fiber is 
measured by the detector and wherein tiie processor further functions to 
calculate a theoretical length of the fiber based on the frequency changes 
measured by the detector and compares that calculated length with tiie known 

20 length ofthe fiber to calibrate the measuring electronics. 

Brief Description of the Figures 

Fig. 1 is a block diagram of a system in accordance with an embodiment 
of the invention. 

2 5 Fig. 1 A shows the overall flow of how one may use an embodiment of 

the invention to scan an object, organize acquired points, fit geometric shapes to 
the organized point, manipulate the fitted geometric shapes, and display the 
resulting manipulated geometric shapes. 

Fig. 2 is a more detailed block diagram of the system of Fig. 1 . 

3 0 Figs. 3 and 3A show the physical arrangement ofthe FDV ofthe Fig. 1 

system, and also shows how the FDV is coupled to the tripod by a fork mount. 
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Fig. 4 shows an example coordinate system relative to the FDV of flie 
Fig. 1 system. 

Fig. 5 is a block diagnun of one embodiment of an FDV in accordance 
with the invention. 

5 Fig. 6 is a block diagram of the optical transceiver of the Fig. 5 FDV. 

Fig. 6A shows a dual mirror arrangement of the scanner shown in Fig. 6. 
Fig. 7 is a block diagram which shows an embodiment of the laser. 
Fig. 7A is a block diagram of an embodiment of the beam expander 
shown in Fig. 6. 
1 0 Fig. 8 shows an embodiment of the duplexer. 

Fig. 8A shows a partially-reflectiiig diq>lexer. 

Fig. 9 shows an embodiment of the window of the Fig. 8 duplexer. 

Fig. 10 is a flowchart that shows calculations performed by the FDV 

DSP. 

15 Figs. IIA and 1 IB showa unidirectional scan pattern and a bi- 

directional scan pattern, respectively. 

Fig. 12 is a block diagnun of an embodiment of the FDV processor. 

Fig. 13 is a block diagram of example circuitry for determining a desiied 
position of an FDV mirror. 

2 0 Fig. 1 4 is a block diagram of an example signal conditioning and tmatgy 

integration circuit of the timing circuit shown in Fig. 12. 

Fig. 1 S is a detailed block diagram of the system of Fig. 1 . 
Fig. 16 shows two windows used to operate the COP. 
Fig. 17 shows a targeting box and a point cloud. 
25 Fig. 18 shows a point ckwd from the surfiwe of a horse sculpture. 

Fig. 19 shows the point cloud of Fig. 18 color mapped with the laser 

Fig. 20 shows a clmid of points from a comer Seature. 

Fig. 21 shows the cloud of points of Fig. 20 and a polygonal lasso used 

3 0 for manual segmentation. 

Fig. 22 shows the cloud of points of Fig. 20 segmented into four 
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subgroups, three subgroups on the surfaces of planes and a subgroup of edge 
points that are not part of the plane. 

Fig. 23 shows the cloud of points of Fig. 20 rendered as a triangulated 

mesh. 

Fig. 24 shows the comer feature of Fig. 20 with planes fit to the groups 
of cloud points. 

Fig. 25 shows a point cloud from the surface of a cylinder. 

Fig. 26 shows a cylinder primitive that was fit to the points shown in 

Fig. 25. 

Fig. 27 shows a cloud of points from the surfaces on a piping system. 
Fig. 28 shows cylinder primitives that were fit to the points shown in 
Fig. 27. 

Fig. 29 shows the completed piping model, after extending pipes and 
adding elbows. 

Fig. 30 shows the resiilt of a corner fit, giving three planes, three lines, 
and a vertex. 

Fig. 31 shows a cylinder primitive in a scene. 

Fig. 32 shows the cylinder form Fig. 31 extended to meet adjacent 

objects. 

Fig. 33 shows a cloud of points from the surface from a variety of 

objects. 

Fig. 34 shows a model containing primitives that were fit to the points 
shown in Fig. 33. 

Fig. 35 shows configuration of a frequency adjustable laser. 

Fig. 36 shows block diagram of conventional FM chirp lidar. 

Fig. 37 shows block diagram of self-calibrating FM chirp lidar. 

Fig. 38 illustrates the relative timing at which a large and a small pulse 
cross a predetermined direshold. 

Fig. 39 illustrates one circuit for measuring pulse energy. 

Fig. 40 illustrates another circuit for measuring pulse energy. 
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Detailed Description 

A. Qv^iyy 

1. Overall Slystem 

Fig. 1 is a block diagram that illustrates the invention in its broadest 
5 aspect. Referring to Fig. 1, a Field Digital Vision (FDV) module 10 includes a 

scanning sensor for scaiming an object 20 and for sensing the position in three- 
dimensional space of selected points on the surface of the object 20. The FDV 
module 10 generates a point cloud 30 v^tch refHesents the sensed positions of 
the selected points. The point cloud 30 also represents other attributes of tte 
10 sensed positions, such as reflectivity, smftce color and texture. 1^ 

A C(Hnputer Grqihics Perception (CGP) module 40 interacts with tiur 
FDV to provide control and targeting fimctions for the FDV noodule 10 soisi^. 
In addition, using the point cloud, the CGP module 40 recognizes gemietrie 
shqies represented by groups of points in the point cloud 30. and the CGP 
15 module generates a CGP model 42 that represents these geometric shapes. 

From the CGP model 42, the CGP module 40 generates a further model usable 
by computer-aided design (CAD) tools SO. The CAD tools may be 
conventional. 

Fig. 1 A shows the overall flow of how one nwy use an embodiment of 
20 the invention to scan an object, organize acquired points, fit geometric Shapes to 

the organized point, manipulate the fitted geometric shapes, and display the 
resulting manipulated geometric shapes. 

2. FDV Module Overview 

25 Referring to Fig. 2, the FDV 10 includes a scanning laser system (lidar) 

210 that scans points of the object 20 and that generates a lidar data signal that 
precisdy rqmaents the position in three-dimensional space of each scanned 
point The lidar data signal for groups of scanned points collectively constitute 
the point cloud 30. In addition, a video system 220, preferably including both 

30 wide angle and narrow angle CCD cameras, is provided. The wide angle CCD 

camera of the video system 220 acquires a video image of the object 20 and 
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provides, to the CGP 40 via a control/interface module 230 of the FDV 10, a 
signal that represents the acquired video image. 

In response to user input relative to the signal that represents the 
acquired video image, the CGP 40 provides a scanning control signal to the 
5 iidar 2 1 0, via the control/interface module 230, for controlling which points on 

the surface of the object 20 the lidar 210 scans. More particularly, the scanning 
control signal provided from the CGP 40 controls an accurate and repeatable 
beam steering mechanism to steer a laser beam of the lidar 210. 

In addition, the narrow angle CCD camera of the video system 220 
1 0 captures the texture and color information, and provides this captured 

information to the CGP 40. 



3. CGP Module Overview 
Referring still to Fig. 2, the CGP 40 is constituted of a data processing 
1 5 system (e.g., a notebook computer or a graphics workstation) and special 

purpose software that when executed conFig.s the CGP 40 data processing 
system to perform the FDV 10 control and targeting functions, and also to 
perform the CGP model 42 generation functions. 



The CGP 40 controls the scanning lidar 210 of the FDV 10 by providing 
a lidar control signal to the FDV 10 that controls M^ch points of the object 20 
the FDV 10 scans. User iiqsut is provided to the CGP 40, which defines what 
portions of the object 20 to scan, and at what resolution. 

25 

ii. Model Generation 
Each data point in the point cloud 30 generated by the FDV 1 0 
represents both distance to a corresponding laser impingement point from an 
FDV 1 0 "origin point" and the angle from the origin point to the laser 
- 30 impingement point. The CGP software conFig.s the CGP 40 computer to 

process the data points of the point cloud 30, generated by the lidar 210 as a 
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result of scanning tbe object 20, to diqilay and visualize the scanned pntiont irf' 
the object 20. More specifically, the CGP software conFig.s the CGP 40 
computer to recognize geometric shapes in the object 20 ("graphic percepttoa**) 
and, using these recognized geometric sibapes, to perform geometry 
construction, 3D model construction, 30 visualization, and database functions 
for automated acquisition or manual input of object attributes, generation of 
plans, sections, and dimensions, data query, and CAD interfaces, and 
networicing options. 

B. I>etail8 

1. FDV Module Detail 

Fig. S is a block diagram of one embodiment of an FDV 10 in 
acG<xdance with the invention. A lidar transceiver 502 includes a lasn, t wiMimt 
optics, receive optics and detector for generating ranging and intensity data. A 
s ca nn i n g system 504 includes dual orthogorud scanning mirrors, galvo motois, 
and encoders for steering the laser beam and determining the azimuth and 
altitude angles of die ]aaa beam finm the positions of the mirrors. A wide 
angle video system 506 generates targetmg video information and a narrow 
angle video system 507 generates color and texture infonnation. 
Control/interface circuitry 230 handles the exchange of data between the FDV 
10 and the CGP 40. 

If the laser beam is quasi-CW, always on widi eidier intensity 
modulation (AM) or wavelength modulation (FM), the distance to the otyect 20 
can be infened by any of a number of techniques mvolving demoduktira at the 
transceiver 502. If the laser is pulsed, the distance to the object 20 is usually 
measnied by the time of flight from the transceiver 502 to the object 20 and 
back. Other laser ntodulation schemes may be used. 

In the time-of-fligiht embodiment, die laser is preferably of the type 
disclosed in U.S. Patent Nos. 5,132,977; 5,386,427; and 5,381,431, assigned to 
Massachusetts Institute of Technology. In particular, the beam generated by 
such a laser has special properties such as being capable of producing pulse 
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widths less than Insec. 

A particular embodiment of the laser which has been used is particularly 
suitable for precision lidar since: 

1 . The short pulsewidth provides high accuracy, since radar theory 
shows that the accuracy is proportional to the inverse of the pulse time. 

2. The laser itself is physically quite small, especially useful for 
portable applications. 

3. It has a diffraction limited beam, which implies that the spot size 
at a distance is not limited by the properties of the light, but only by the 
quality of the optics used to colUmate or focus it. 

4. Since the wavelength is quite short (S32nm), and Rayleigh range 
is inversely proportional to wavelength, the beam can be kept small over 
a large distance interval. In fact with a 1cm exit aperture, the beam will 
remain less than 6mm over SOm. 

In one prefeired embodiment, the laser beam is directed by the 
orthogonal scanner mirrors to a laser impingement point on the sur&ce of the 
object 20. The range can be determined by any of a number of conventional 
lidar techniques. For example, the "time of flight" of a laser pulse to travel irom 
the laser to the surface of the object 20, and then back to the detector, is 
determined. The range is determined based on the constant speed of light, with 
appropriate adjustments being made for atmospheric factors. 

A system in accordance with the present invention can provide high 
ranging accuracy at high acquisition rates. For example, at 1 00 m ranges, a 1 
mm accuracy can be achieved on a single shot basis, with anywhere from 1 000 
to 5000 data points being acquired per second. 

In other embodiments, a chirp lidar may be employed. The essential 
component of a chiip lidar can be modulated with a linear change of wavelength 
over time. Thtis. the wavelength of the light emitting from the laser will be 
given by X(t) = k(t-to ) + Afl. In practice, such a laser is commonly 
manufactured by making a composite of two materials, a typical laser gain 
media such as NdYAG (35 10), and substance which can have its index of 
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refractioa altered electrically such as Lithium Niobate (3S20). (See Fig. 35) 
This effectively changes the length of the laser cavity, and hence alters tite 
emitted wavelength. Commercially available lasers can be modulated iq) to 
about 100 GHz with a voltage modulation of about 1 kV, so the frequency of 
5 the light will be approximately f(t)»k(t-to ) + ^. 

Referring to Fig. 36, in a typical FM chirp system, a portion of tfie light 
emitted by the laser 361 0 is sampled and recombined at the beam splitter 3630 
with the light returning from the target 3620. Since the light is delayed by the 
amount of time to contact the target and return, the light returning from the 

10 target will have a lower frequency than the light v^ch is sampled from the 

laser. Hiis difference will be appamat in the ou^iut of detector 3610 wiiich -~ 
measures the intensity of Hat combined beams. If the frequency nunp of ttj^js 
exactly linear and the laser has a coherence length much greater than the 
distance to tile target, combining the beams wiU imduce a constant measunNt . 

15 frequency from the detector wfaidi is proportional to the range: f- k| *d-(-d^ 

Chirped Yag lasers as described above have a coherence length of about 20 km, 
but the chirp is not linear, and this has severely limited the accuracy of existing 
FMchiiplidars. 

Referring to Fig. 37, a huge improvemoit in accuracy can be realized by 
20 adding a system to calibrate every range measurement A fiber is prepared 

which has a partial reflector 3771 on one end, and a nearly total reflector 3772 
on the other. Now a portion of the light emitted by the laser 371 0 is sampled 
and recombined at beam sfditter 3740 with tiie light returning from the target 
3720, and dw intensity is measured by a detector 3760. An additional sanqile of 
25 tiw beam emitted from laser is sampled by beam splitter 3730 and 
introduced into the fiber at the paitiaUy reflected end 3771. Thebeam 
propagates a fixed distance down die fiber and reflects off the end face and is 
recombined widi tlte beam which is reflecting off the partially reflecting fiu% 
3771, and is measured with a second detector 3750. The linearity of the chirp is 
3 0 then measured by determining the deviation from a constant frequency of the 

output of detector 37S0, and this information is used to coirect for the effects of 
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the nonlinear chirp in the output of detector 3760, which corresponds to the 
target range measurement. 

Referring to Figs. 3 and 3A, in the embodiment, the FDV 10 is 
physically housed in a box 330 made of metal or other suitable housing 
5 material. The box 330 is suspended from its side panels by a fork mount 

mechanism 310. The fork mount system is supported on a turntable 340, and 
the turntable 340 is mountable on a tripod 320. Using the fork mechanism 3 10, 
the FDV 10 can be rotated horizontally ("azimuth rotation") and vertically 
("elevation tilt", or "altitude"). Generally, a position of the tripod 320 is 

1 0 referred to as a "setting" or "positioning"; the rotation and tilt of the FDV 10 in 

the fork mount 3 10 is referred to as "pointing" or "orientation". A "view" is 
generally associated with a given setting and orientation. 

The fork mount system 3 1 0 preferably includes high accuracy aidmuth 
and altitude rotation measuring devices (e.g., conventional theodolite-type 

1 5 optical or electronic encoders) to provide precise rotation and tilt data of the 

FDV 10. This feature can allow the automatic integration of scans taken from 
the same tripod 320 setting, but with a dififeient orientation of the FDV 1 0. In 
the event these devices are not used, and for scans taken from different settings 
and orientations, these scans can be integrated using techniques described later 

20 in this disclosure. 

It should be noted at this point that, while conventional surveying 
instruments should be leveled prior to operation in order to operate properly, 
this is not a requirement of the FDV 10. This is due to the novel nwthods of 
this invention as embodied since its own internal coordinate system and the 

2 5 procedures utilized in its software that take advantage of its method of acquiring 

position data. The system, however, does have the ability to be leveled and to 
be used in a manner similar to a traditional theodolite. The Cartesian 
coordinates of the FDV 10 in the example embodiment are shown in Fig. 4. 
Referring still to Figs. 3 and 3A, in one embodiment, two orthogonal 

3 0 mirrors of the FDV 1 0 provide a field of view of approximately 40* by 40° 

("Field of View", or "View" is defined as the maximum size of the area 



wo 97/40342 



PCTWS»7ff793 



-19- 

projected by the laser maximum deflections of the beam in degrees). The field 
of view can be inoeased or decreased by resizing the minors and certain parts 
of the optical train. The fork mount described above is utilized to allow 
pointing of the FDV's 40' x40' field of view anywhere over a projected sphere 
5 thus afiFording a wide range of flexibility in imaging large objects or groups of 

objects from the same setting. Other mounting methods may be used to 
accomplish the same purpose. 

High accuracy and repeatability electronic encoders read the rotational 
angles of the orthogonal mirrors, and the readings of the minor rotation anglet 

10 are precisely timed to coincide with the range reading. Preferably, the systen is 

Class U FDA eye safe. A first embodiment has t6nun spatial accuracy over a 
range of t SO m. In another embodiment, autofocus capability and 5<6 
picosecond electcoaics are included, which extends the system's range accurate 
to 1 1mm and ± 1mm spatial accuracy over ^ 50m. The range (and accuracy) of 

15 thesystemcanbesignificanUy influenced by the choice of eye safety 

classification selected, but diese limitations are not inherent limitations of the 
invention itself 

The following is a description of tfic key components of preferred 
embodiment of the FDV 1 0. A block diagram of tiie optical transceiver 502 of 

20 tiie FDV 10 is shown in Fig. 6. The optical transceiver 502 transmits an optical 

pulse to a spot on object 20, and receives a reflected optical pulse from die 
object 20. Given die constant speed of light, tiie optical transceiver calibrates 
the distance to tiw spot on tiie target 

Refemng to Fig. 6, tin laser 602 fires an optical pulse which lasts less 

25 than 250psec, in response to an external command provided from a laser 

controller 604. The laser 602 produces a pulse, preferably at a wavelcngtfi of 
S32nm, witiiin 100-300^sec after an external signal emanating from a digital 
signal processor which provides central control of real time events. The time 
delay is a complicated fimction of recent laser history and environmental 

3 0 c(»iditions. This function is not con^letely known at present However, a 

software algoritiim, which is described elsewhere, is employed to estimate tiie 
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time delay with adequate accuracy for the required measurements. 

The laser beam output of the laser 602 is transmitted through a beam 
expander 606 that is focused to adjust the size of a light spot that will eventually 
impinge upon a point on the object 20. The focussed optical pulse is then 
5 transmitted through a duplexer 608, which is an optical system for aligning the 

outgoing optical path with the incoming optical path. The duplexer 608 directs 
a significant first portion of the light energy of the outgoing optical pulse to a 
spot on the object 20 via a scanner 614, but a second, mvch smaller portion, of 
the light energy of the outgoing optical pulse is directed to a receiver telescope 

10 610. The portion ofthe outgoing optical pulse that propagates to the object 20 

impinges on the spot on the object 20, and some of the energy of the optical 
pulse is reflected off the object 20 in a direction back to the duplexer 608. The 
returning optical pulse is directed by the duplexer 608 to a receiver telescope 
610. which focuses the received energy onto a detector 612. The detector 612 

1 5 converts the received optical pulse energy into electrical energy, and the ou^wt 

of the detector 612 is a series of electrical pulses, the first (which is generated 
by the detector 612 in response to the second, small portion, of the transmitted 
pulse not directed toward the object 20) occurs at a short fixed time (i.e., fixed 
by the length ofthe optical path through the beam expander 606, the duplexer 

2 0 608 and the receiver telescope 6 1 0) and the second of which occurs as light 

energy returns firom the object 20. Both the second, small portion of the 
transmitted pulse not directed toward the object 20, and the return optical pulse 
reflected from the spot on the object 20, are fvovided to the timing circuit 6 1 6 
which calculates the time of flight to the spot on the object 20. The range to the 

2 5 spot on the object 20 can then be readily calculated fh>m the calcuUted time of 

flight 

Fig. 7 is a block diagram which shows an embodiment of the laser 602. 
The heart of the laser system 702 is a conventional laser chip 702 that include 
two bonded crystals coated with antireflective dielectric coatings on their faces. 
30 The laser chip 602 is pumped with a solid state diode 704 operating at 

808.S nm t3nm. The ou^t frequency of the diode punq) 704 is adjusted by 
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changing its temperature widi a thermoelectric coolo' 706. The temperstun of 
the diode pump 704 is measured with a thermistor 708, and the measured 
tonpmture is fed back into the diode power supply 710. The required ' ~ 
temperature varies with each individual diode, but it typically ranges from 2(f 
5 to 30" C. 

The output power of the diode pump 704 is typically 1 watt, launched 
into a 100 ^m core glass fiber. When continuously pumped, the output of the 
crystal laser 602 is ^proximately 35mW average power at 1 .064 \xm, which 
corresponds to 2.4 ^U pulses lasting about 280 psec at a repetition rate of 1 SkHz. 

10 The multimode fiber is preferably terminated by an SMA905 solid brass 

c<»uiector. with the crystal of the laser chip 702 glued to one end of die 
connector with an optical resin. Thus ensures adequate thermal dissipation ftpm 
the crystal of the laser chip 702, keeping the crystal 702 within the tempenilie 
range required for most efficient operation. 

15 A piece ofKTPfiequency doubling crystal 712 is held within a few 

millimeters of the fiice of the Waet chip crystal 702. This provides an ultimate 
output from the laser 602 having a 12 mW average power at S32nm, which 
conesponds to O.S^iJ pulses lasting approximately 218 psec. This ultimate 
output from the laser 602 is nearly dififraction limited (Le., one which has 

2 0 theoretically minimum divergence, given a specific wavelength and waist 

diameter), with an apparent waist diameter of S6\jan. 

Embodiments of the invention vdiich meet FDA Class 11 eye sale sy^on 
design specifications are potentially more commercially viable. In cider to 
meet this specification, the maximum energy per pulse that can be transmitted at 

25 S32nm is .2 |iJ. With this restriction, the average power transmitted is largely 

dependent on the pulse repetition rate, and is given by the following table 



30 
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CLASS 


ENERGY PER 
PULSE 


AVERAGE 
POWER 


REPETITION 1 
RATE 


I 


2tx3 


.39 


1.9SHZ 


IIA 


.2mJ 


3.9 fiW 


19.5 Hz 


II 


.2mJ 


1.0 mW 


5kHz 


IIIA 


.2^J 


5.0 mW 


25 kHz 



In one embodiment of the invention, the beam expander 606 is entirely 
conventional (e.g., Melles Griot model number 09LBM013, 1 Ox beam 
expander). The transceiver 502 has cross axis accuracy which is proportional to 
5 the size of the laser beam impinging on the intended target. The base design of 

6mm accuracy has a simple beam ejqnnder. The laser 602 can be coUimated 
with a fixed lOx beam expander 606 which has an aperture of < 1 cm to produce 
a beam whose l/e^**^ beam width is less than 6mm over a range of 50m. 

Fig 7A shows a further embodiment 750 of the beam expander 606 that 

1 0 includes features which allow the system of the invention to measure ranges at 

an accuracy of q>proximately 1mm at SOm. This is because the impingement 
spot on the object 20 of the laser beam expanded by a conventional beam 
expander is coUimated. and produces a spot of no more than 6mm over a SOm 
range. However, a beam can be focused through a 50mm apatuK to a spot of 

15 size no more than 1mm over a SOm range - but the spot will be much larger at 

odier ranges. Thus the beam expander 750 of a system having 1mm accuracy at 
50m includes a movable optical element 752 vAdch can change the size of the 
focused spot. Additionally, the beam expander 750 includes an adjustable 
aperture 755, and means for controlling the adjustment, so that the distance, 

2 0 from the laser, over which the beam stays at 1mm in diameter remains 

approximately constant. The minimum diameter spot produced with a 
diffraction limited lens of focal length f and diameter D is do = 2DJD. The 
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Rayleigh range of the focused spot, which is the depth of focus of the beam, is 
given by b - 2nS)o'/A. - 2iif X/D*. Thus, if £13 is held constant, the de]Mh of 
focus will not be a function of tiie range of the focused ^t, f. 

As the beam focus is changed, die elements ^uld stay sufficiently 
5 aligned so as to prevent the beam from changing direction by more than a 

fraction of 1 nun at SOm, or this will appear as an error in the placement of the 
point in space. In order to minimize diis beam wander, a linear servo motnr 754 
(see Fig 7 A) is employed for controlling the position of the focusing 
mechanisn, and a tcansducer ixovides position feedback. The lens 752 is 
10 mounted in an annular ring 753, wduch prevents it from rotating or misafigning 

while it is being translated. ~ 

Duplexcr 

An embodiment of the diq>lexer 608 is shown in Fig. 8. The optical 
1 5 system of the di4>lexer 608 is conFig.d such that the outgoing beam from the 

beam expander 606 to the scanner 504 is coaxial with the return beam reflected 
from the object 20. Thus, only one scanner 504 need be provided. In the 
embodiment of the diq>lexer shown in Fig. 8, a window 802 is provided, with a 
50% beamsplitter 804 attached over the window 802. When an optical pulse is 
2 0 transmitted from the laser 602 and through the beam expander 606, the pulse 

impinges upon the beam splitter 804. Most of the light energy of the pulse is 
reflected o£f the beam splitter 804 and is passed to the scanner 504, but some of 
Ae optical pulse proceeds through the beam splitter 804 and inqMnges iqjm 
low reflectance beam Mode 806. Due to the low (but non-zero) leflectanoe of 
25 the beam block 806, a small fraction ofthe optical pulse hitting the beam block 
806 returns to tiie beam flitter 804 and is reflected into the receiver 610. 

Moreover, as an optical pulse returns from the object 20, since only the 
central portion of the return pulse is obscured by the prism 804, most of the 
light impinging on the window 802 makes its way to the receiver 610. 

30 

Partially reflecting dtqilexer 
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Refecring now to Fig. 8A, for the 1 mm accuracy embodiment, a 
partially-reflecting duplexer 850 is employed. With this duplexer. a fraction of 
the light pulse provided from the beam expander into a beam stop 652 and 
reflects off the duplexer window 850 to the receiver telescope 61 0. The 
5 remainder of the light pulse proceeds to the object 20. Most of the return light 

pulse from the object 20 continues on through the window 850 and is collected 
by the receiver telescope 6 1 0. The window 850 is AR coated on the receiver 
side, and partially miirored on the laser side. The entire window 850 is used to 
steer the outgoing beam, since a 50 mm aperture is required to focus the spot to 
10 1 mm at SO m. The partial reflectance is chosen in view of the laser 

transmission power and the applicable eye-safe classification level. For 
example, if the laser transmission power is four times the allowable level of the 
applicable eye-safe level, then the partial mirroring is diosen to reflect 25% and 
absorb 75%. 

15 Referring now to Fig. 9, in the 6 mm embodiment, improved efficiency 

can be achieved in collecting the return c^ticai pulse if only the center of the 
window 802 is coated 904 to reflect the outgomg pulse, and the remainder of the 
window 802 is anti-reflective coated 906. In this way, the return optical pulse is 
not reflected out of the receiver by the part of the window 802 that is 

2 0 antireflection coated 906. 

Preferably, the laser 602 emits a strongly polarized beam so that the 
reflective coating 904 can be optimized to have slightly different reflection 
coefficients for the two planar polarizations (20%-S and 30%-P). In such an 
embodiment, the power of tiie beam im|nnged onto the object 20 can be fine 

2 5 tuned merely by physically rotating the laser body. 

Receiver Telesc<^ 

Referring again to Fig. 6, after the returning pulse has passed through 
the duplexer 608, it is collected by the receiver telescope 610, v^ich optimizes 
30 the amount of signal provided to the detector 612. The receiver telescope 6 1 0 

may be a simple SOmm aperture lens. The lens is preferably selected so that the 
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variation in pulse energy mtoing the detector 612 does not change as a function 
of the distance to the object 20 over the range of distances for which the 
instrument is designed. A multiple element lens can be designed to minimtze 
the variation in received pulse energy as a function of range somewdiat mem 
5 effectively than a single element lens. That is, at the greatest expected (fistaoce, 

the focal length of the lens is such that all the incoming light, vsliich is 
effectively collimated since it is generated by a point source in the far field, is 
focused to completely fill the detector 612. As the object 20 becomes closer to 
the telescope 610, the spot of return light becomes larger than the detector 612. 

10 The power incident on the detector 612 increases as the square of the distanoe 

fiom HtB telescope 610 to the ot>|ect 20, iqi to the nuximum expected distanoe. 
Moreover, die power returning from the object 20 decreases as the square of the 
distanoe fimn the telescope 610 to the object 20. Thus, in practice, diese two 
effects ^iproximately cancel each other. This minimizes the variation in opHSxtti 

15 power incident on the detector 612 over the range of anticipated use. In the 

1 mm option, the receiver optics can be improved in some cases by using a two 
element, adjustable focus, Newtonian telescope (e.g., similar to the 1 mm beam 
oqiander). 



20 Detector 

The detector 612 converts optical pulses to electrical pulses wfaidi am 

be processed by the elapsed time measurement electronics (timing circuit 616). 

In one embodiment, the detector 612 is an avalanche photodiode (APD) whh 

greater than 1 GHz electrical bandwidth. In addition to the time between the 
25 start and any stop pulses, the intensities ofall the pulses are recorded. The 

intensity information is used to make a correction to the range measurement 

derived from the timing information. 

Scanner 

^0 Thc8camer504m^beconvnitional. The scanner 504 directs the 

outgoing pulses from the duplexer 608 to a desired position on the object 20 and 
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directs the incoming return pulse into the receiver telescope 610. The scanner 
504 directs light to the narrow field cod camera S07 to collect color and texture 
in the inunediate vicinity of the scanned laser points, which provides for precise 
registration of color and texture obtained with the lidar acquired point geometry. 
5 In one embodiment, the scanner 504 includes a dual mirror arrangement (see 

Fig. 6A) for beam steering, although any conventional high accuracy and 
repeatability beam steering mechanism may be employed. The dual mirror 
arrangement includes two mirrors which arc rotated on orthogonal axes by 
moving coil motors. These motors have an integral position decoder widch has 
1 0 angular repeatability of less than 1 microradian. The mount for the scanners is 

integrally formed with the supports for the laser and other optics. This system 
provides 40 degrees of optical motion in both altitude (elevation) and azimuth at 
several Hertz. 



15 Electronics 

A. Timing Circuit 

Another embodiment of the scanner 504 mechanism consists of a single 
mirror rotating about a central axis, mounted on a rotating turret. In this 
configuration, the physical coordinate system would be spherical, with the faster 
2 0 (due to the less inertia) mirror providing the elevational angle and the more 

slowly rotating tiirret providing azimuthal motion. A system such as this could 
provide a field of view of more than 90 degrees in a vertical plane and a full 360 
degrees in a horizontal plane (both phmes being relative to some chosen scanner 
coordinate system. 

25 

Electronics 

Ranging Electronics 

The function of the ranging electronics is to compute the range from the 
FDV 10 to the object 20 based upon the electrical output of the detector 612. 
30 Severalpossiblemethodsmaybeused,includingademodiilator inthecaseof a 

quasi-CW modulated laser system. For the preferred time of flight embodiment. 
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an interval timer (timing circuit) measures the relative time interval between an 
initial (start) pulse reflected directly into the receiver 610 by the duplexor 608, 
and the pulse reflected off of the object 20 back into the receiver 610. 

Reflectivity Electronics 

In many cases, it is useful to know not only the position in space of a 
point on the object 20, but also know the reflectivity (at some paiticular 
wavelength) of that point The reflectivity electronics measure the amount of 
light reflected from the object 20 into the receiver 610 and detector 612. Thb 
data can be used to provide corrections to the range information as well as the 
information on the material and/or flnish of the surface of the object 20. 

Digital Signal Processor 

A digital signal processor integrated circuit controls all the time critical 
functions of the FDV - scanner control, laser firing. It also provides fiat 
floating pouit computation capability f«r making geometry corrections, 
calibration corrections, and video lens corrections, and video compression. The 
digital signal processor is intenupted at regular time intervals, typically about 
10 usee. At each of these time intervals, a check is made to see vrhai real time 
calcuUdiona ave outstanding. 
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Scanner Control 

The electronics for the scanner are a simple precision PID controller 
which are driven by a digital signal from the DSP. When driving this system 
quickly, there is noticeable lag in the ability of the scanner to follow the driving 
signal. However, the controller circuit does not have an error signal output. An 
external precision analog differential amplifier provides an error signal (the 
difference between the command signal and the actual displacement), which is 
sampled by the DSP at low resolution. The DSP then computes the exact scan 
position by computing the sum of the command signal and the error signal. The 
advantage of this method is that it requires only a low resolution A/D converter 
and a precision D/A convoler, rather than a far more expensive precision A/D. 

The distal signal processor generates the trajectories for the analog 
scanner controller, and makes measurements of the difference between the 
desired trajectory and the actual position. It predicts the time at which the laser 
pump is turned on so that the laser will fire at the desired angle. These 
predictions are made at regular time intervals. Fig. 1 0 is a flow chart that shows 
the calculations performed at each time interval. 

Trajectory computation 

The user defines areas within the view of the scanner that are to be 
scanned, and indicates the density of points to sample within the scanned 
region. There are several scan patterns which can be used, and these require a 
specific pattern of mirror movement, known as the trajectory. The objective of 
picking a good trajectory are the conflicting needs of doing the move quickly 
and accurately. Accurate movement requires minimum torque, vdiich would 
otherwise defoim the q)paratus. This limits the speed with which motions can 
be made. At equal time increments, a calculation is performed to determine the 
current position of each mirror. The particular calculation used depends upon 
the type of scanmng employed. 



Raster scanning 
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When the desired scan field is a polygon, one of two raster scanning 
patterns is tised. In the first, scanning is uni-directional (i.e., always proceelsi 
from left to right, or right to left, on parallel lines). Fig. 1 1 A shows such a 
unidirectional scan pattern. In between scan lines, the scan mirror retraces to 
the beginning of the next line without making any range measurements. The 
retrace can proceed quite quickly since no measurements are being made during 
the retrace. 

A slightly more efficient means of raster scaiming is bi-directional, in 
which scanning is also performed on the retrace. Fig. 1 IB shows such a 
bi-directional scan pattern. This is not as efficient as it might seem because tlie 
retrace time is used for other calculations, and because the resulting scan paltem 
is not as regular. 

Both raster scanning methods require traversing a straight line in ttie 
m in i m u m time, starting at zero velocity and aiding at zero velocity. The txsque 
qiplied to the mirror is proportional to the angular acceleration, which must 
zero at Aeb^inning and end ofthe scan since the mirror is at rest. Itcanbe 
^wn that the trajectory that nuikes such a minimum energy move between two 
points is given by the sum of a straight line and a full cycle of a sin. However, 
this is closely approximated with much less computation by the minimum 
degree polynomial , with boundary conditions p(tO>»pO, p'(tOH>, p"(tO)-0, 
p(tl)-pl, p'CtlH), and p"(tl)=0 which is the fifth order polynomial: p(t) - (p, - 
pjt- (6t^ - IStf + 10) + ft. where tf = (t - t,y(t, - 1^. 

Spiral Scanning 

A disadvantage of raster scanning is that since the speed of the trajectory 
is varying, the scanning efficiency is not optinud. A spiral pattern can achieve a 
constant speed trajectory which pennits a uniform point distribution. 

Seeking 

In addition to scanning a range image, the system is capable of 
perfimning a number of fimctions viibkh are common in surveying. The scanner 
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can be made to search for important features, or locations of high reflectivities. 
This allows the system to perform normal surveying functions by finding a 
target whose location is iqiproximated identified, and reporting its exact angles 
and position. 

5 

Angle Calibration 

The capacitive encoders in the moving coil motors have tremendous 
repeatability, but relatively poor accuracy. A number of calibration activities 
need to be continuously performed to ensure system accuracy. 

1 0 Before use. each scanner is calibrated over its complete range of angles. 

At a number of discrete temperatures, a map is created and stored of the 
measurements of apparent angles for thousands of accurately measured points 
using an external lesolver that is traceable to NBS standards. The DSP linearly 
interpolates between dwse measured points on every angle measurement. 

1 5 Preferably, the accuracy of angle measurement is improved by 

determining scale or offset errors in the encoder during operation. 
Commercially available scanners can drift significantly with environment 
changes. This results in a shift in the effective zero and full scale range of the 
angle measurement, while maintaining the overall shape of the calibration curve 

2 0 obtained by making careful laboratory measurements before the scanner is 

installed in the system. The environmental effect is reduced by providing a 
means for determining when the scanner is at a known and repeatable angle. In 
one preferred embodiment of such a system, two optical references which are 
fixed with regard to the case of the instrument are aimed at the back of each 
25 scanning mirror. There are a A^ety of mechanisms for providing the optical 

reference, but in one preferred embodiment, a pair of autocollimators are aimed 
at a reflective surface on the back of the scanning mirrors and will provide a 
highly repeatable measurement of when the mirror is normal to the axis of each 
autocoUimator. Each autocollimator gives a reference angle to within 

3 0 approximately 10 ^rad. Periodically, the scanner is moved under computer 

control to the position at M^ch the mirror is closes to being normal to the 
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autocoUimator axis, and the iQ>paient angle is measured. The measurements we 
compared with the measurements taken when the scanners were calibrated, and 
a linear correction is calculated and applied to every subsequent measurement. 

In an alternative embodiment, a pah- of mechanical stops is provided just 
past the normal range of motion of the scanning mirror. Periodically, the mirror 
is driven until it touches a mechanical stop. Then, the scanning mirror is driven 
with a known current, which corresponds to a known force. The mirror airives 
at equilibrium at a very repeatable position, and this is used to calculate a linear 
correction to the mirror calibration curves. 

Range Calilxation Fibos 

The timing circuits have a certain amount of offiet and scale drift whit 
time and temperature, and a provision has been included to compensate for 
these variations. When an optical pulse is emitted fixmi the laser 602 a small 
amount of the energy is sampled by a beam splitter 81 0 and introduced into a 
single mode optical fiber 830 by focusing the beam using a lens 833 on the fiber 
face 831. The other face of the fiber 832 is arranged so that the beam which 
conies out of it is coUimated into a beam which enters the lidar receiver 610. 
The fiber can either be produced so that its length does not vary with 
temperature, or its variation in length with tenqjerature can be accurately 
characterized. Whm a single mode fiber is used, die vaiiatim in propagatim 
delay wiU be less than a few picoseconds and the pulse shape emitted by tiie 
fiber will be nearly identical to the pulse shape going into the fiber. 
Periodically, the timing circuits are used to measure dw propagation delay 
through the fiber, and conesponding adjustments are made to the range 
measurements taken from external suiftces. 

The fibers can be manuftctured so that the end at which the pulse is 
launched 833 and from which the pulse is emitted 834 are partially reflecting. 
When this is done, the pulse enters 833 and is propagated to the opposite end 
834, at which point only some of the encigy is released and the rest returas to 
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the first end 833. Again, a fraction of the light is emitted, and the rest reflected, 
which eventually is emitted into the receiver. This process repeats until the 
remaining energy in the fiber Mis to negligible levels. The result is a sequence 
of pulses, commonly 3-10, being iqiplied to the receiver, yMch have delays all 
5 repeatable to within a few picoseconds. Periodically, the timing circuits are 

used to measure these pulse trains from the fiber, and corresponding 
adjustments are made to the nmge measurements taken fh>m external surfifices. 



Range Walk Calibration 

1 0 The lidar system measures the range of surfaces by timing the delay 

between the laser pulse being emitted and returning from the surface. This 
delay is measured electronically by imposing a sample of tfie outgoing pulse, 
and the return pulse on an optically sensitive electronic detector 612 embedded 
in the receiver 610. In one, embodiment, the electronic timing circuit measures 

15 the time between when theoutgoing pulse exceeds a set direshold voltage, and 

when the return pulse exceeds the same voltage. The outgoing pulse will be the 
same intensity within a few percent However, many surfaces vary greatly in 
the amount of light that will be reflected. The result is that the apparent relative 
time for two pulses which occur at the same range but have different intensities 

2 0 may appear to be at different ranges. The measured time for a small pulse 3810 

to first exceed the threshold level will be later than the measured time for a large 
pulse 3830 to exceed the same threshold, even v/hen the pulses return from 
objects at the same range. Thus highly reflective objects or objects at distances 
of maximum transceiver sensitivity will appear slightly closer. This creates an 
25 apparent"naige walk" as a function of mtensity. The range walk can be 

corrected if the Aape of the optical return is always the same and the energy of 
the return is known. The extremely repeatable shi^ of ttie pulses generated by 
the passively Q-switched microchip laser makes this possible. 

Part of the timing circuit estimates the energy in each detected pulse. A 

3 0 table of corrections is maintained to improve the range estimates. Two different 

circuits have been employed to make a measurement of the pulse energy for this 
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purpose. Tlw first is a gated iniegntor, the gate bein^opm at the begum^ 
the piilse, and closed at the end. The signal is applied to a compaFator 3920 
\^ch closes the switch 3930 when the signal exceeds a selected level, and 
closes it when the signal falls below the same level. The signal is also applied 
5 to a delay 3910, and the output of the delay goes through the switch 3930 when 

it is closed, and is applied to the integrator 3940 over the period of time the 
switch is closed. The delay is chosen to compensate for the time lag in the 
comparator and switch. Whoi the pulse is cmnplete, the value of the integraior 
is sampled by an analog to digital converter 3950. The second consists of a 
10 imegntor with a time omstant scaled to the width ofthejnilse 4010, followed 

by a peakdetectcMT 4020 wAidi has a time constant much longer than the pulse 
width. The output of the peak detector is sampled shortly after the pulse is 
detected. 

Periodically, the timing circuit is used to measure a sequmce of pidtes 
IS which have been delayed by the single mode fibers 830 used to calibrste the 

offset and scale &ctors associated with the time circuits. Additionally, the 
intensity of these pulses are varied over a broad range by a variable attenuate 
820. By altering the amount of light coupled into the fiber, the energy of the 
detected pulses can be varied over the dynamic range of the receiver, at one 
20 particular time delay. The intensity and the measured time delay valura produce 

a mi^ of the range walk correction requued for each intensity, and this 
correction is q>plied to subsequent measurements. This correction can provide 
accuracy of 1mm over the dynamic range of the instrument, paiticttlarly as t 
result of the great repeatabOity of die laser pulse waveform. This fimction ia 
25 thai used to correct die measured range ofextemal surfaces as a fimctioii of 

light intensity returned from those surfaces. 

Geometry Calculation 

The ou^ of the FDV after a range scan consists of points in spherical 
30 coordinates widi respect to a coordinate system in die scanner. However, the 

raw data consists of minor angles and time intervals. The DSP computes the 
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spherical coordinates of the scanned points by taking into account scanner 
geometry (minnor thickness, optic axes, mirror offsets, etc.) and all the 
appropriate calibration adjustments. 
Laser Control 
5 delay prediction 

The digital signal processor is responsible for controlling the firing of 
the pulsed laser, but it can only do so indirectly. The processor has control of 
the timing for starting the pump diode, which causes the passive q-switch to fire 
after saturation has occurred. However there is a variable delay between turning 
10 on the pump and having the laser fire. The delay is a function of junction 

temperature, which in turn is a complex function of ambient tempoature and 
recent history of laser firing. The delay generally ranges between 100-300usecs. 

Fortunately, it is i»imarily necessary to know the scanning minor angle 
at the precise moment the laser fires. After the laser has bera fired just a few 
1 5 times, the pump delay does not change quickly if die firing rate does not change 

quickly. As a result, accuracy of a few microseconds can be achieved by 
estimating the next pump delay to be the same as that in the previous firing 
cycle. The digital signal processcM* measures the pump delay by reading an 
internal counter when the pump is started and when the laser actually fires, 
2 0 causing an interrupt. Since the interrupt latency is less than a microsecond, this 

becomes the timing accuracy to which the pump delay can be measured. 

A more sophisticated dynamic model of the thermal properties of the 
laser could lead to slightly enhanced scaiming pattern regularity, but is probably 
equally limited by the time resolution of the i»rocessor interrupts. 

25 

Firing control 

Given a time vs. angle trajectory for a scanning axis, w(t). a desired 
angle to fire the laser, and an interrupt interval Dt, the decision to fire tfie laser 
amounts to computing the time at which point the pump diode is started. 



30 
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Computer Control 

The FDV is designed to poform under the control of a remote host 
computer ^ch contains gnqihical controls for a user to sqiecify areas to he 
scanned. The remote machine controls the FDV through a bi<lirecti<mal soial 
5 byte stream, which is effected in any of a number of media: Ethernet, EPP 

parallel port, serial port. A processor in the FDV is assigned the task of 
decoding messages, and scheduling the required activity. Fig. 12 is a Mode 
diagram of the FDV processor. 

1 0 Host Conununications IntnfiKe 

The host machine acts as a master, sending a well defined message 
protocol to command the FDV. When actions are convicted, the FDV resqMmds 
with data and status information. Among tfie actions vtdiich can be requested 
are: 

15 Point the scanner 

measure a distance 
range scan a box 
fire the laser n times 
take a video image 

20 

Scanner Control 

RefoTing to Fig. 13, in normal operation, each scanner in the dual tttom 
system requires a 16 to 18 bit digital woid to set the desired position, which is 
applied to a preciaon digitd to anatog converter to create a voltage pcofMfti^ 

25 to the desired position. However, tfiere will be some error between the position 

commanded by the output of this converter and the actual position of the 
scanner, which is reflected by the output of the position encoder. A precision 
diiferaice signal is generated, and the difference is measured to 12 bit accuracy. 
Hiis provides an economic method of making 18 bit position measuranents 

3 0 vMlc only unng an ineiqiensive 12 bit convoter. 

Commeroially avaihdile galvo scanners have microiadian repeatability. 
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but have relatively poor scale and offset performance, particularly over 
temperature. A calibration mode has been incorporated into the system to 
permit making measurements at two precise angles, and using the two measured 
data points the offset and scale drift of the scanner can be calculated. 
5 Two methods have been developed for this purpose: an optical and a 

mechanical means. In the medianical method, the scanner shaft is gently placed 
against one of two mechanical stops, and the current in the scanner controller is 
adjusted to a specific value, which provides a known force. The position signal 
is adjusted imtil there is no position enx}r, and this gives the calibrated position 

10 measurement In the optical method, two autocollimators are aimed at the back 

of the scanner mirrors, ^ch have also been polished and minor coated. When 
the scanner mirrors are exactly aligned with one of the collimators, the output 
from the split photodetector in the autocollimator is balanced. By placing the 
scanner in each of these precise angles in turn, an offset and scale correction for 

15 the scanner encoder can be calculated. 

Timing Circuit 

The purpose of the timing circuit is to provide the relative time between 
the start pulse and the stop pulse, in picoseconds. There are two subsystems in 

20 the timing circuit: a signal conditioning and energy integration circuit (an 

embodiment of which is shown in Fig. 14) , and a time interval analyzer. Both 
communicate directly with the DSP. Initially, systems have been produced with 
a commercial timing instrument, the Stanford Research Systons SR620 time 
interval analyzer. The interfiace to this instrument is through an IEEE488 

25 interface. In a jHcferred embodiment, the communications inter&ce to the 

Stanford Research Systems SR620 time interval analyzer is IEEE488. 

A custom time interval measurement circuit has been developed which 
utilizes a separately patented interpolation technology. The circuit employs a 
clock, typically operating at >100nihz, vfbich is used to make a coarse count of 

30 lOnsec intervals between stop and start pulses. Additionally, there is an 

interpolator which divides each lOnsec coarse count into 1000 smaller 
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inciements, providing lOpsec resolution. This system has approximately Spaec 
jitter. DifTerential time measurements can be made with less than 20psec RMS 
error, wliidi corresponds to about 3mm. This circuit conununicates with the 
DSP using a dedicated serial bus, and employs a packet protocol: the DSP aims 
5 the circuit by sending a single byte. When the timing circuit completes its task, 

it sends a sequ^ce of bytes vAdch represent both the time delay between start 
and stop pulses, and the intensity of each pulse. J 

Laser Firing 

The DSP has three lines for laser control: one starts the laser pump, the 
second indicates that the laser has fucd, and the diird indicates that the return 
pulse from a target has been detected. When the laser fires, the DSP samples 
the analog pulse amplitude signal. This happens typically within 1 ^sec. 

15 Video 

For targeting, the user is provided on the host a video representation of 
the scene fiom which he can choose a portion to be range scanned. Inmost 
cases this will correspond to the scene rendered in ambient illumination. 

20 Capture 

One way the video is captured is by using the scanner to aim single 
sensitive detector across the scene with the laser turned oflf. This permits 
acquiring an image which has very accurate spatial alignment with subsequent 
range scans. However, inuige cqiture can be quite slow m compariacm to 

2 5 commacially available cameras. 

A second approach is to utilize standard commercial CCD video 
cameras to acquire an image. One CCD camera with a wide angle lens is 
aUgned with the range scanner with as smaU an ofl&ct as possible. A second 
camera with a 5 degree field of view is phiced so that its optic axis is coaxial 
30 with the transceiver. Thus, a much smaller field of view is accessible through 

the scanner, and can be scanned with the same resolution as the transceiver. 
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This allows targeting small or distant objects. 
Alignment 

The wide angle lens introduces a fish-bowl effect in the image captured 
5 by the CCD sitting behind the lens. Straight lines in the world are not straight in 

the image. This distortion increases with the distance from the coiter of the lens. 
This distortion is removed by comparing the image the camera produces v/iaai 
aimed at a carefully designed and printed calibration target image. The 
difference in the anticipated image and the recorded image provides the 
1 0 information needed to waip subsequently acquired images to eliminate the 

distortion. 

Compression 

Each video image is compressed prior to transfer. Currently we are using 
1 5 JPEG standard image compression. It is relatively fest, and creates reasonably 

small compressed images for conununication. Another desirable feature is that 
the algorithm operates on blocks, which permits us to do interieave image 
capture, alignment, compression, and transmission in parallel — significantly 
enhancing throughput. 

20 

Point Video 

A second camera, with a narrow field of view (e.g., approximately 5°) is 
placed such that it is coaxial with the scanning laser beam. The field of view is 
adjusted so that the pixel resolution is apimximately the same as the voxel 
25 resolution of the lidar system. The camera can be operated ^le the laser is 

activated. When Hm is done, a small group of pixels will be illuminated by the 
laser, and the ccntroid of these pixels will correspond to the point vMdi would 
be measured by the lidar. When a video image is captured, it can be mapped 
onto a surface which is estimated by a lidar scan. 

30 

Computer Graphics Perception (CGP) Software 
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RefCTTUig to Fig. 15,theCGP40isasoftwaresystemtf>atninsona 
CGP Computer 1500 and communicates with the FDV 10. The CGP 40 nms on 
many different types of computers, including l{q>tops and workstations. The 
CGP 40 operates on a computer 1500 with a suitable display device 1510, s^j^ 
5 as a color graphic display terminal, a suitable character input device 1 520, aiidi 

as a keyboard, and a suitable pointing device 1530, such as a mouse. The 
software can use any number of standard 3-D gnq>hics rendering libraries to 
interactively present the acquired 3-D data in a window on the display device. 
The portion of the CGP 40 user interface that involves 3-D view manipidad^ 

10 and data projection into a window is handled by the 3-D libraiy. 

The CGP 40 performs real time 3-D data acquisition and modeliti^ ui iSx 
field. The COP's 40 functionality includes high level FDV 10 control. tHgeiing 
and data acquisition; display and visualization of scanned pouits; surface 
segm«itatton and fitting; manual 3-D model construction; 3-D visualizaiksi; 

1 5 interaction with part and model databases; and the ability to export data in 

standard data exchange formats to other CAD systems for fixrther processing. 
The integration of hardware and software, as described here, enables major 
improvements in productivity and quality in the overall process of three 
dimensional modeling of reality. 

2 0 With reference to Fig. 1 A, the data acquisition and modeling ixocess 

divides into the following steps: FDV 10 control, point acquisition, 
segmentatim, geometry fitting, modeling by manipulating the geometry, natne 
r^istiation with or witiiout warping, model annotation, and geometiy dSafAvjf 
and query. 

2 5 Witii reference to Fig. 16, the foregoing operations may be petfonned in 

at least two gra|^c diqilay windows. One window 1610 displiQrs a video 
image of tiw target scene used to define regions to be scanned by tiie FDV 10 
while die otiier window 1620 displays an interactive 2D projection of the 3-D 
model consisting of the scanned points and constructed surface geometry as 

30 well as otiier information about tiw scene. Additional windows may be used to 

provide multiple views of tiie data. In addition, tiie CGP 40 provides additional 
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windows for controlling the FDV 1 0 hardware and for setting and displaying the 
status parameters of the system. 

Scan Control 

Referring to Fig. 1 5, prior to using the integrated hardware/software 
system the FDV 10 is positioned to point in the direction of the objects 20 of 
interest. 

Scan control is the process of indicating which portions of the scene that 
are visible to the scanner arc to be scanned. Different parts of die visible scene 
can be scanned at different densities, since simple geometric objects, such as 
planes, cylinders and spheres can be accurately modeled with a Surly low 
number of scan points. Therefore, the region in fiont of the scanner is often 
cultured in multiple scans, rather than in one high resolution scan. Only regions 
with high levels of detail need high resolution scans. 

Referring to Fig. 1 7,'one of the means of scan control is the use of a 
video image 1710 of the scene acquired from the FDV 10. Using a pointing 
device, such as a mouse, one can indicate the region to be scanned by any 
number of methods, such as dragging out a rectangle 1720 on the video image. 
The CGP 40 instructs the FDV 10 to measure the range of whatever object 
exists at the center of the user specified targeting region to assist in specifying 
the scan density, since the angle between points is determined by both the 
desired density on the sur&ce and the distance firom the scanner. A means for 
specifying the desired scan panuneters, such as a dialog box, is provided and 
allows the user to specify the scan parameters in a variety of ways, including 
point density, point spacing, or total number of points in each of the vertical and 
horizontal directions. 

The CGP 40 then translates the region and scan resolution information 
into a set of conunands for the FDV 10. These conunands are commimicated to 
the FDV 10 using a means of communications, such as a TCP/IP network 
connection, and the acquired data is also returned to the CGP Computer 1 500 
using the same means. 
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Additional scans at dififeient densities can be initiated in the same wi^, 
or one can use previously scanned data points rather than the video image to 
specify new scan regions. If the view of the scanned data is oriented so that it is 
exactly aligned with the scanner direction, then a scan region can be indicated 
by methods such as dragging out a rectangular box. When the data is aligned to 
the scanner in this way most of the 3-D information is difficult to see, therefore, 
the software can display the points with the intensity of the returned laser light 
at each point color mapped as described in the next section. The intensity 
informaticm is oflen sufficient to identify objects in the data window, so Unti 
new scan regions can be defined. Alternatively, the user can model and/or eo!or 
some of the objects in the scene to help locate regions of mteiest in the wiliitiw. 
Using the data window to define new scan regions avoids any parallax ernes, 
since tiie view is aligned with the scanner. 

Scan control can also be achieved by using the pointing device to move 
tte laser beam and highlight points in ttie actual scene. Any number of methods 
could be used to describe the desired scan region by moving the laser beam and 
identifying points of interest by a user action, such as clicking a mouse buttmt 
Methods could include operations such as: indicating a bounding box by 
moving the laser to diagonally opposite comers of the desiied scan regions; 
indicating the top, bottom, left and right bounds of the scene; mdicating a 
sequence of pouits that represent the bounding polygon of the scan region; 
mdicating the cater of the scan region and using other means, such as dialog 
boxes to describe the extern of the desired scan region. 

Point Acquisition 

Still wilfa reference to Fig. 17, the data returned by the FDV 10 consist 
of the coordinates of the points and their intensity values. In one preferred 
embodiment, the scanning is peifonned in such a way that the data returned Ues 
in an ordered grid of three dimensional points 1730. Viewed from the scanner, 
these points appear as a regular rectangular grid, much like a bitmap. However, 
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each point consists of its coordinates in three-space as well as the intensity of 
the reflected laser pulse at that location. 

Each point returned is displayed in the data window 1620 as it is 
trananitted by the FDV 10. The CGP 40 lets the user interactively change the 3- 
5 D view of the data while the data is arriving to get a better idea of the spatial 

layout of the data. Also, to help visualize different features in the data, of the 
CGP 40 can allow each point to be color mapped from the intensity of the 
reflected laser pulse at that location. A scan cloud 1 8 1 0 is shown in Fig. 1 8 
representing the surface of a horse sctdpture. Instead of having all the points in a 
1 0 single color, as shown in Fig. 1 8, one can map different laser return intensity 

values to dififerent colors, and produce a multicolored scan field 1910 as shown 
in Fig. 19. The intensity color mapping provides considerable extra surface 
feedback to the user and this is useful in both targeting and modeling, as 
described later. 

1 5 The ordered grid of points generated is referred to as a scan field. 

Multiple, possibly overlapping scan fields may be gathered and simultaneously 
displayed in the manner described above. The data structures within the CGP 
40 maintain the list of scan fields, so each data point is always associated with a 
scan field. The scan fields usually contain data points from the surfaces of many 

2 0 different objects, so they need to be partitioned into smaller groups of points, as 

described in the next section. 



Segmentation 

Segmoitation is the process of grouping together points diat were 
25 scanned from the surface ofthe same object The points fi»m a single object 

may be a small portion of a scan field, or may occur across multiple scan fields. 
The segmentation process may be manual, as described below, or automated, as 
described later in the auto-segmentation sectiim. 

Referring to Fig. 20, the first step of the manual segmentation process is 
30 to select one or more scan fields 201 0 that contain scan points on the object of 

interest. Selecting one or more scan fields can be performed by any of the 
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conventional means, such as using a pointing device, possibly togeOxa with 
keyboard keys. Selecting a scan field selects all of the points in die scan field. 
The group of points resulting bom this s^ fonn a pool of candidate points that 
can now be trimmed to remove points on other objects. Each point in the pool is 
5 initially nuuked as selected, and the operations described below can be used to 

toggle the point states between selected and deselected. 

Refexring to Fig. 2 1 , scan points 20 1 0 from a desired object surface can 
be cut out using one or more lasso qieiations firom possibly difEnent views. The 
user can manipulate the view direction as needed to give a clear view of Uie l. 

10 desired subset of the point pool. The user then uses the pomting device to dfsw 

a polygonal lasso region 2110, wdiich divides die screen into two regions: the 
interior and exterior ofthe lasso polygon. The following operations aie 
supported: mark all points in a region as selected and all of the other points «» 
deselected, mark all of the points in a region as selection without af&clii|g 

IS other points, and maik all ofthe points in a region as deselected without 

affecting the other points. The lasso <^)eration may be repeated as numy times 
as necessary to refine the selection, possibly changing the view ofthe scene 
between lasso operations. The user can the cut the currently selected set of 
points out to form a new point set The new point set acts like a scan field in 

20 that it is, and can take part in the fitting operations described in the next aectkm. 

In Fig. 22, three new groups of points 2210, 2220 and 2230 have been created 
using the manual segmentation method described here, and some points near the 
intenection of die planes are left fiom the original cloud of points. 

25 

Geomrtiy Fitting 

In one preferred embodiment, the CGP 40 can contain many geometric 
primitives that can be used to simulate the actual surfaces ofthe objects 
scanned. The geometric primitives include any number of standard graphics 
30 primitives, such as triangulated meshes, planes, cylindeis, spheres, torii.line«, 

and points. The simplest tana of geometry fitting mvolves using a ttiangubted 
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mesh to connect the scan points to show the surface features of the objects 
scanned. The scan cloud 1810 in Fig. 18 can be meshed 2310 and rendered as 
shown in Fig. 23. Since the scan data are acquired in a regular grid, it is simple 
to create a triangular mesh by connecting neighboring points. The user can also 
5 set discontinuity tolerances in depth and angle to avoid meshing adjacent points 

separated by more than the specified threshold. Breaking the mesh in this way 
provides a more realistic looking sur&ce, referred to as a shrinkwrap surface, 
because artificial mesh surfaces at occlusion edges do not occur. A wide variety 
of known mesh operations can be qiplied to the resuhing mesh, such as 

1 0 smoothing (noise reduction) and mesh simplification (to reduce the mesh 

density in smooth areas that do not require a fine mesh grid). Mesh vertices may 
also be colored with infoimation such as intensity. 

As stated above, the CGP 40 includes many standard geometric 
ivimitives. Before fitting the points to sudi objects, the point clouds must be 

1 5 segmented as described above. Once segmented, each group of points represents 

a single surface that can be fit by a geometric object. The fitting can be guided 
by the user, who may know the type of sh^ to be fit. For instance, after 
scanning the comer of a room it is clear to the user that the points on a wall can 
be fit by a plane and the points on a pipe can be fit by a cylinder, so the user can 

2 0 request the fit of a specific object It is also possible to semi-autonute this 

process to identify which sh^ best fits a particular pomt group. 

Fitting a plane to a set of points is a simple problem that has many well- 
known solutions. The extent of the patch used ui the CGP 40 to rqnesent the 
plane can be determined by the convex hull of the points in the plane. For 

2 5 instance, the three point groups 2210, 2220 and 2230 shown in Fig. 22 can each 

be separately fit to planes 2410, 2420 and 2430 as shown in Fig. 24 using any 
available fitting algorithm. 

Many standard approaches are available for fitting more complex 
shapes. In one preferred embodiment, two phases are involved: a parameter 

3 0 estimation phase to get a starting point, and an optimization phase, where the 

parameters are varied to minimize the overall error. The total error is ftie sum of 
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the squares of the distance betwem eadi scan point and the nearest po 
surface of the object being fit. The optimization phase uses oonvatimuit 
optimization methods to jeducc the enor between the object, as defined bf 1P 
parameters, and the data given by the scan points. 
5 A cylinder fitter can convert a cloud of points 25 1 0 as shown in Fig. 25 

into a cylinder object 2610 as shown in Fig. 26. All fitted objects, including the 
cylinder, refoence the original points diat were used to fit the object. The user 
may choose to view the resulting cylinder 2610 or the original points 2510, or 
both, at any time. Using manual or automatic segmentation methods, it is ... 

10 possible to convert die scan clouds, 2710 in Fig. 27, representing many ' 

cylinders into the best fit cylinders 28 10 shown in Fig. 28. Once each cylindbr's 
diameter and axis are established, it is possible to manually or autonuriicaltiradd 
elbows 2910 in Fig. 29 to complete the modeling process. 

A cylinder is described by five parameters: a normalized vector 

15 describuig die cylinder axis (two independent parameters), the radius, and two 

additional parameters that are used to locate the line of action of the cylinder 
axis in space. The length of die resulting cylinder can be detennined by 
projecting the scan points onto die cylinder axis and noting the extreme vahm 
of this projection. 

2 0 Two novel mediods for estimating i^linder parameters are mqilemoitod 

in one preferred embodinient The first way to find initial parametCT estimaliesi 
for a cylinder is to find approximate sui&ce normals, as described ill the aiittK 
segmentation section. If all of thenomuds are setto unit length, then they can 
all be consider to be vectors torn the origin to a point on the sur&ce of the unit 

25 sphere. If one uses each normal vectm- and its to accumulate a group of points 

on the unit s|dme, then one can fit a plane through the resulting group of points. 
The resulting plane normal is roughly parallel to the cylinder axis. Given the 
cylin<ter axis and die plane from the previous step, one can project the scan 
pomts onto die plane. The projected points will be well described by a circle in 

30 diis plane, since die plane is normal to die cylinder axis. A best fit circle can be 

calculated using die projected pomts on the plane to give an estimate of die 
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cylinder radius. The center of the circle on the plane can be converted to a 3-D 
point to give a point on the cylinder axis. 

The second way to estimate the cylinder parameters is to fit the set of 
point to a quadric surface, vMch is described by the implicit equation: 

F(p) = 0 = c,pf + CjpI + Cjp] + c,/»,ft + c,PtPj + c^PjP^ 

+ C,p,+C,Pi-¥C,Pi+Cn (I) 



where p = {Pi,Pj,Pi} is a point on the quadric surface. 

10 

One can then take c„ « -1 since the equation is implicit and perform a 
least squares fit with all of the data points to determine the other nine 
parameters. After determining the best fit quadric sur&ce Cor a given set of 
points the next step is to find a point actually on the new surface (p,) that is in 
15 the vicinity of the other points. This is achieved by finding the centroid of the 

scan points (p^ ) and then finding the closest point on the sur&ce of the quadric 
to give p, . The normal to the surface at point can be determined using: 

N,=D,p, + D, (2) 
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Two unit vectors, n, and U2 , are then found such that they are normal to 
both each other and . These vectors form a basis for the sur&ce at the point 
under consideration, and additional vectors on the surface can be found as: 
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v„ = cosa •¥ Ujsina, 0^a^2fF (3) 



The unh jvinciple vectors v„ are then found by detennining the rotatiim 
5 a that satisfies: 



.(n,xD,v.) = 0 



(4) 



There are two solutions to this equation that yield the oithog<»ai unit 
principal vectors V, and v, . The surfiM:e curvatures k in these two principle 
duections are then given by: 




(5) 



For cylindrical sur&ces one of the principal curvatures will be near zoo, 
and the radius of the cylinder is the reciprocal of the absolute value of the 
nonzno curvature. A mediod to determine the radius (r) and axis of the 
cylinder has been described, and only the location of the axis needs to be 

determined. A unit surfece normal can be calculated as n = N,/|N4 "Ow*** 
of the normal can be adjusted so that it points towards the interior of the 
cylinder by ensuring that n • (p, - p,) > 0 since the ccntroid of the points lies 
on the interior of the cylinder. A point on the axis is then given by p, + r a . 

These starting parameters can then be used in a niinimization process to find the 
best fit parameters fisr tfie cylinder. 
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The novel method described above for curvature estimation using the 
quadric surface formulation is further used in a novel way for automated object 
type determination. If the points being fit are well represented by a plane then 
both principal curvatures will be near zero. If the points being fit are from a 
cylindrical surface then one curvature will be near zero and the other will be 
nonzero. If the points are fix»m a sphere then both curvatures will be nonzero 
and their magnitudes will be approximately equal. Combining the automatic 
detection of object type and the auto-segmentation algorithm, described later, 
allows the CGP 40 to have a novel method for automatic fitting of many objects 
that occur in typical scanned scenes. 

A further use of the curvature estimation is sphere fitting, which is 
achieved by using the quadric surface approach to approximate the radius and 
location of the center point, and then using a four parameter (center point and 
radius) minimization to reduce the error between the sphere model and the 
measured points. The novel method described above for finding a point on the 
axis of a cylinder is also used in the preferred embodiment to find the center of a 
sphere. 

The segmentation techniques disclosed above can be used to create a 
variety of useful fitting tool based on combinations of the previously described 
shapes. For instance, a comer, consisting of an intersection of three planes 
which may or may not be orthogonal, is a very common feature to scan. 
Knowing that the specified point group contains three intersecting planes, such 
as the points 2010 in Fig. 20, the points are automatically segmented, using the 
technique described later, into three subgroi^is of points that each lie on 
separate planes. Then, any available plane-fitting algorithm can be used to fit 
tiie planes to the scan points in each group. Unlike the more genoal auto- 
segmentation algorithm described later, if one knows that the comer object 
consists of three planes, tiie fitting algorithm need not try to fit cylinders, 
spheres or other objects and check which produces the best fit, only planar fits 
are required. Refening to Fig. 30, the comer fitting tool not only fits the planes 
3010, 3020, 3030 but can also intersect tiiem to complete the comer. As shown 
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in Fig. 30 additional useful information can be given to the user by int«rifejS§|| 
lines 3040, 3050, 3060 at the intersection of pairs of plains, and a vertex point 
3070 that represents the location of the comer. This vertex point is much more 
accurate than a single scan point, because each plane is fit using many data 
5 points, and the vertex is generated by intersecting the planes. The above novel 
ntethod for automatically creating a comer with its intersection lines and vertex 
can be used as a tool of the CGP 40. 

Eadi object stores information about die quality of the fit, so that ibe 
user can query the object and examine the noean, standard deviation, and wo^ 

10 errors. Knowing the accuracy of the FDV 10,theCGPortheusercaBllieii : 

decide if an error has been made during the fit Errors can arise when the MlpPg 
type of primitive is fit, or when extnmeous points that were not actually acanaed 
from the desired surface remain in the data set In addition, the objects store ; 
their geometric parameters so the users can query for radius, length or other 

15 values of interest 

In addition to the class of general object fitters, which are given close to 
no initial information other than the pouits to fit, there is a class of fitters Had 
can take advantage of foreknowledge about objects in the scene. An area in 
which such foreknowledge exists is that of the construction industry, where 

20 parts used follow standards ofdimensions and design. For exanqsle, the 

mteinal diamder of pipes fimn a particular nuuiu&cturer may come in five 
different sizes: 4", S", 6.5", 8", and 10". This infomution typically ivsides id 
tables that describe relevam attributes of these parts. The cylinder fitter can ttke 
advanti^e of the information in these tables to significantly reduce the solution 

25 space to be searched: the fitter need only search for solutions involving 

cylinders of one of those diameters. Anotha* way to use such table lookups is to 
have the fitter come up with a general solution, then match against entries in the 
object tables to find the entiy with the closest parameters. For example, a pipe 
fit by a cylinder with 7.8" diameter would be matched against the 8" entry in the 

3 0 table from the example above; the user (or fitter) then has the option of refitting 

an 8" cylinder to the pipe, or accepting the 7.8" cyluider. Yet another use is for 
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the user to manually select a specific entry (or collection of entries) fiiom the 
table and tell the fitter to use its parameters in the fit, v«^ch also reduces the 
fitter's solution space (which can decrease the time taken). 

5 Modeling 

The fitting of geometric primitives, as described in the previous section 
does not usually complete the modeling process. It is often the case that only a 
portion of the objects surface is scanned, such as one end of a cylinder or a 
portion of a wall, and fiirther operations are required to complete the 3-D model. 

1 0 Modeling is the process of completing the construction of the 3-D model given 

some fitted geometric primitives. 

Many common CAD operations such as extension, intersection (mutual 
extension) and trinuning are available in the CXiP 40. For example, the cylinder 
31 10 in Fig. 3 1 does not initially extend to the plane of the floor 3120. In Fig. 

15 32 the cylinder 3220 has been extended to the floor plane 3 1 20. These simple 

operations enable rapid completion of portions of the model from the objects 
created by geometry fitting. For instance, given three planar patches that were 
fit fiiom scan data near a comer, it is easy to mutually extend the three planes to 
complete the comer feature. 

2 0 Object extensions can be accomplished in several ways. One way is to 

select the geometric object to be extended and tell it to extend to a subsequently 
sdected object The nature of the extension is determined by both the type of 
object to be octended and the second object selected. For example, a cylinder 
extends the end closer to the second object along its centeriine until its end 

2 5 intersects with the infinite plane defined by the second object's geometry (in the 

case of a planar patch, the infinite plane is that of the patch's plane, and for a 
cylinder, the infmite plane is that containing the centeriine and as orthogonal to 
the extending cylinder as possible). 

Another way is to nutke use of object handles, which are nodes that the 

30 user can grab. These handles are tied to an object*s definition (position, 

orientation, and size) vAtsrc iqipropriate, and by moving a handle, the object's 
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definition changes accordingly. Again, taking the cylinder as an example, die 
same extensi<»i described above can be accomplished by grabbing the haui&t on 
the Old to be extended, and then moving the handle (and extending the cylinder) 
to the desired position. A handle's motion depends on the part of the object 
5 with which it is tied; a handle on the centerline of the cylinder is constrained to 

move only along that centerline, v^le a handle on the boundary of a planaff 
patch is constrained to move only inside the plane of the patch. For some 
objects, handles may be inserted and removed, changing the definition of die 
shape of the object (for example, handles on a planar patch have a one-to-one 
10 com^KHidence to vertices on the planar patch's boundary). Other handles 6tn 

provide rotational control over an object. The control of handles is intenMSili 
and dynamicaUy updates, so the user can see the mteimediate results of the 
redefinititm. 

A new operation, called merging, has been developed to allow diffenmt 
1 5 portions of a single object surface to be joined to form a single object in the 

CX5P 40. It is often the case that one's view of an object is obscured by other 
objects in fiont of it. For instance, the view of a back wall in a room may be 
divided into two pieces because of a column in the foreground. A scan of the 
region will result in different groups of points on the particular object. If aiito- 

2 0 segmentation is used, as described later, rather than manual methods wfaa« « 

user knows that the points belong to the same object, then separate point groups 
would be formed. Each pomt groiip would then be fit to a separate object, 
resuhing in multiple pieces of the same surface. Hie two objects in the CGF 40 
that are known to be on the surface of tfie same feature, such as the two planv 
25 patches of wall obscured by a column, can be merged to form a single object. 

Each object stores a reference to the data points that define it, so when a merge 
request is received a new geometric fit is performed on all the underlying data 
points that were part of the constituent geometries to achieve the best overaU fit. 
This novel nuthod of mcreasing the accuracy of object fittmg is used in the 

3 0 merging operations of one preferred embodiment of the invention. The 
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characterisUcs of the two primitive objects that were merged do not affect the 
outcome of the merge; only the underlying point positions are considered. 

Using the manual or automatic methods, the user can take a cloud of 
points 3320 from the surfaces of many objects, such as the points on the 
pyramid 3310 shown in Fig. 33 and convert them to a set of geometric objects 
3420, such as the planar face of the pyramid 3410 in Fig. 34. The modeled 
scene shown in Fig. 34 accurately represents the features of the original objects 
that were scanned, and allows measurements to be made between any particular 
locations in the modeled scene. 

Scene Registration 

The initial position of each scan point is described in a local coordinate 
system whose origin is that of the FDV 10, and whose axes are fixed relative to 
the FDV 10. Therefore, scan fields taken without moving the FDV 10 are 
inherently registered in that all the scan points use the same coordinate system. 
However, if the FDV 1 0 is moved between scanning operations, additional 
effort is required to transfonn the data to use a shared coordinate system. This is 
a six parameter rigid body transfonnation of a data set, involving three 
translation and three rotation parameters, and is easy to apply to the data once 
the transformation is known. 

A novel process is used to register the scan fields from different FDV 1 0 
positions. The novel registration process requires the user to identify pairs of 
points, lines or planes in two different scenes that represent the same feature. It 
is also possible to use different features, such as the back and front ftce of a 
wall, if the user knows the offtet between them. The planes and lines are 
converted to point pairs, as described below, and the process operates entirely 
on point pairs. The points used for registration may be actual scan points or may 
be constructed points, such as a comer at the intersection of three planes. 

Given a set of point pairs, the registration process searches the set of 
candidate points for three pans that are not colinear. Using the three point pairs, 
one can construct the transformation required to convert the coordinate system 
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in one view to that used in the other view, wluch in turn can be used to 
tnuisfonn the scan points to all share a single coordinate system. For 
convenience, the process will be described m terms of the first data set 
remaining fixed while the seccmd data set is tiansfonned to use the coonlinate 
5 system of the first. The process worics equally well if the user fixes the ati^M 

data set and transforms the first. 

The first step is to apply a rigid body translation to the second data to 
make the first pair of points coincide in terms of their x, y and z oompoaaOis:- 
The second step is to rotate the second data set about its first poim until tihelbes 

10 fonned by points one and two in both data sets are colinear. The third stq) it to 

rotate the second data set about the line established in the previous step uRti the 
planes defined by points one. two and three in both data sets are coplanar. 

Once an initial estimate is made one can use all die point pairs and an 
error minimization method to reduce the sum of the squares of the distances 

15 between each point pair. 

In order to use the point registration method described above, the COP 
40 uses a novel method to convert lines, planes, and planes with of&ets to sets 
of point pairs. Whenever a nonzero ptene ofl&et is present the new points 
introduced are shifted in the seccMid scene to a position where they will match 

2 0 exactly with the corresponding points in the first scene. The replacement of 

planes and lines with points makes it simple to write the error function for 
minimization, since only point errors are involved, rather than mpiiiw ««h 
distance eirora simultaneously. 

In replacing planes and lines, one can only introduce points that tte at 

2 5 location relative to the user specified objects, since the origins of the two ditt 

sets are different For instance, introducing a new point pair in a plane at tiw 
location closest to the origm would not result in points that actually match in 
space, since the origin is arbitrary. However, introducing a point pair at a plane- 
line intersection will give matching points in the two data sets. Some pairs of 

30 o^ects, like parallel lines, should not be used to introduce new points so an 

aiigutar tolerance. caUed ATOL below, is used to ignore poor object pairs. 
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ATOL is initially set to ten degrees but other values can be used to generate 
fewer or mofe artificial point pairs as needed. The point pairs are introduced in 
the following order: 

For all plane-line pairs where the angle between the line and plane is 
greater than ATOL, introduce two new point pairs. The first new point is 
inserted at the intersection point of the line and plane, and the second point pair 
is inserted alcmg the line of action at a fixed distance away from the first point, 
here taken to be the minimum of the line lengths in the two views. 

For all pairs of planes and points, introduce a new point pair on the plane 
such that the plane normal passes through the new point and specified point. 

For all plane pairs whose normals are at least ATOL qiart, generate a 
new line segment along the intersection of the planes and make the line 
segments length equal to the minimum extent that any plane has along the line. 
The new line segment has no direction, but has both length and position 
information. After this step the planes are no longer needed. 

For all pairs of lines and points, introduce a new point on the line at the 
location where it is closest to the specified point. 

For alt pairs of lines separated by an angle greater than ATOL, introduce 
four new pairs of points. The new points are the ends of line segments along the 
original line of action, but centered on the location of closest approach of the 
two lines. The distance between the new line points is equal to the minimum 
length of the line segment lengths along that line of action from the two data 
sets. After this step the lines are no longer needed. 

The result of the plane and line replacements as described above is a set 
of point pairs that retains the direction information associated with the original 
planes and lines. The augmented set of point pairs can then be used for the 
registration process that is described above. 

After registration of the two scenes, primitives fiom the two individual 
views which represent the same physical object can be combined using the 
merging technique described previously. In particular, matching i^anar patches 
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lepcesenting the same surface can be combined into one extended planar patch. 
Similarly, pieces of matching cylindrical surfaces can be merged to fonn a 
single cylinder. 

5 Warping Data Sets 

The registration process dewribed above is a rigid body transfomution 
that does not modify the relative locations of objects within either data set. After 
registration, most of the point, line or plane pairs that were identified will still 
have small errors, since the minimization process reduces the total mean square 

10 enor. A iwvel method is presented that allows the user to force the identified 

pairs to ex«:tly match by deforming the scene volimies. 

As with any measured data, there is some level of error associated 
eadi scan point location. The nugnitude of the enor associated with a pdnt 
location will vary with the measuring technology used, but some error will 

15 always be present. Since the data under considoation here describes surbce 

features of objects, the data errors will manifest themselves as surface 
irregularities. For example, a set of points acquired from an actual planar 
surface may tK>t all lie on a plane, but will have some small scatter away fpom 
the real plane location. Calculating a best fit plane through the set of measured 

2 0 points may not give the real plane location or orientation due to the enors in die 

point data set 

The errors in recovered feannes, such as planes, cause errors in the 
relati(Xidup8 between the recovered objects as well. For instance, if data is 
collected fnm two planes that have an exactly ninety degree angle between 
25 than, die best-fit planes generated from the data points may not be exactly 

ninety degrees apart. Similarly, cylinders that were parallel in the real scene 
may result in best-fit cylinders that are not parallel after fitting fiom the scanned 
points. These inconsistencies in the recover^ features, that occur due to 
measurement errors, will appear whether the data points are collected from a 

3 0 single scan position or are a union of scans from a variety of diffoent positions. 
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The lack of fit problem may actually grow if several diflerent sets of 
scan data are registered using a relative system. If a series of sequential scans 
are collected, and each scan is registered with respect to some recognizable 
sequence of data points in a previous scan, then the absolute errors in each scan 
5 may grow. If at the end of the sequence of scans the locations of features are 

exactly known, then one must adjust the scanned data points so that they fit the 
known locations. In surveying both the 2-D closure problem and the 3-D 
benchmark matching problems are similar in nature to the problems described 
above. In the surveying closure application, when one surveys a sequence of 

1 0 locations and arrives back at the starting location one typically finds that 

through cumulative measurement errors the starting and finishing locations are 
not at exactly the same locatiotL The closure error, which is the distance 
between the starting in finishing locations, is distributed using well known 
surveying techniques throughout the other data points collected such that the 

15 first and last end points meet after the correction is made. Similarly, when 

surveying benchmarics of known location arc introduced into a surveying data 
set the data set must be adjusted to accommodate the known benchmark 
locations. Both the closure problem and the benchmark matching problem can 
be solved by the method described here since they can be described in terms of 

20 displacement constraints. 

The novel method described here to correct location errors in measured 
3-D data sets and distributes the errors throughout die pomt sets by applying 
solid mechanics {mnciples to a volume suRDunding the data points. The method 
provides a technique for satisfying a wide variety of displacement constraints on 

2 5 3-D data sets and also distributes the measurement errors throughout the data 

sets. The process of deforming the data sets to achieve these goals is called 
warping. The displacement constraints can be specified in terms of both control 
points, whose absolute coordinates are known in space and do not move, and tie 
points, which represent the same location in two or more data sets, but whose 

3 0 absolute location is unknowiL One can describe constraints involving more 

complex objects, for instance, line segments by specifying two points, and 
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planes by specifying three points. In this way an entire fiunily of constFaints can 
be specified and applied to groiq» of objects such as points, lines and planes.' 

All constraints include an ofibet tenn that may be zero or nonzoo. A 
zero offset between points indicates that the points should share flie same fiittl 
location, but does not prescribe where that location would be. If one of these 
points were a control point, with a known absolute position, then it would not 
move and the other point in the constraint would be forced to move to the same 
location to satisfy the constraint. In all cases, the final location would result 
from the energy minimization process that is associated with a solid medbanics 
solution. A non-zero offset between two points indicates that the points «mi;:fi]be 
a certain distance apart after the warping process is applied. ±t: 

If dw constraint's objects are lines or planes (rather than points), ifiettiD 
addition to the ofiset one can specify an angle between the objects. Using this 
ftmily of constraints types, one could specify any number of relations between 
features m one or more data sets. The invention works whether a single or 
multiple 3-D data sets are involved, and the constraints can be between olsfecfs 
in the same or di£ferent data sets. 

The solution of the constraint satisfaction problem begins by registering 
each of the data sets, as described in the previous section, so that all of the data 
shares a single coordinate system. Next, solid mechanics theory is applied tii 4 
volume surrounding the points in each data set to satisfy the displacement 
constraints. The warping method operates on one or more volumes that are 

created so that they sunound all the points in a given data set. Each of these 
volumes is considered to be made out of deformable materials whose ptopertin 
are specifiable. An isotropic material can be used with an elastic modulus of fine 
and a Poisson's ration of aero. The solid mechanics solution finds the mtnimmn 
energy displacement pattern that satisfies the specified constrains. 

One can picture each of these volumes made out of a flexible material. If 
one anchored a volume to prevent rigid body motions and then prescribed a new 
location for a point on the interior, one could envision a distortion of the volume 
that involves not only the point of interest but also the rest of the volume. In 
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reality, the constnunts themselves can be used to anchor multiple volumes 
together. Mechanics principles allow one to determine the minimum energy 
deformation of the volume that satisfies the stated constraints, which mimics 
what would actually happen to a real defotmable object subjected to the same 
5 constraints. 

In a particular embodiment, the warping method uses principles of solid 
mechanics to deform a volume containing points of interest in order to satisfy a 
set of constraints applied to the data points. Not only are the constraints 
satisfied, but the effects of the initial location errors are spread throughout the 
1 0 volumes operated on. 

The finite element method is used to apply the principles of solid 
mechanics to the volumes enclosing the points. The volume is discretized into a 
set of points or vertices and a set of elements that connect to the vertices. Four 
node tetrahedral elements are used to discretize the volume. 
15 The fiist step of the process is to collect the set of constraints that apply 

to one or more data sets. During this phase, one must identify constraints that 
are to be satisfied by the warping process. These constraints include the 
identification of points that represent the same physical location in different data 
sets (tie points), Uke the comer of a cube, which are to appear at the same 
2 0 location when the warping process is completed. Some of the tie points may not 

be points that were scanned in the original data set, but may be constructed from 
groups of other points. For instance, if one had a series of points that 
represented three planes intersecting at a comer, then one could fit three planes 
to the points, and use the resulting comer point as a tie point. The constraints are 

2 5 specified in twms of pairs of objects, such as points, lines and planes, as well as 

the desired offset and angle between them. The two objects involved in the 
constraint can be contained in a single data set or can occur in different data 
sets. Within a single data set, one could specify that lines or planes remain 
parallel, or that the distance between two points be a specified amount. Between 

3 0 multiple data sets, one could write similar constraints, or indicate that the 

features seen in two data sets represent the same object. One might also know 
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the actual location of some points very accurately (benchmarks) and consttain 
points in tiw data set to lie at the known locations. Using these bendunatk 
points to anchor dififerent points in the data sets enables the closure proUetn to 
be solved, since the data sets will be waiped so that the measured dirtapotait 
5 move exactly to the desired control point locations and the errors in the data set 

will be smoothed over ail volumes. 

The second step in the warping process is to register all the data seta 
involved, as described in the previous section. 

The third step in the warping process is to select a volume that surrounds 
10 the region ofmterest, and describe the volume in terms ofa set of new points. 

The region that can be scanned by the FDV 10 is called the view volume and is 
shaped like a pyramid, with the tip of the pyramid located at the origin of tlli 
scanning device. A pyramidal shape can be used to bound the view region for 
the purpose of warping, and the pyramid is easily described by five points, tu^ng 
15 the same ooordmate system as the data points. These new points do not become 

part of the data set, but are used in the warping process. The convex hull of 
these points represents the new volume surface, and should enclose all the data 
points on the interior. This operation is performed separately for each data set. 

The fourth step is to mesh each of the data volumes. Meshing involves 

2 0 filling the volume with finite elements that leave no gaps and that do not 

overlap. The finite elements span between the points or vertices that have been 
defined on the volume boundary and those that are involved in conatndnts m 
the interior. The points in the data set do not all need to be included in the 
warping process, only those that are used in constraint specifications and dioie 
25 that define the volume boundary need to be used. The elements in the initial 

mesh may be of poor quality due to thcff shape. Ung sliver elements, for 
instance, are known to give poor results in finite element analysis. Therefore, 
the meshing process is actually iteraUve. New points are inserted into the mesh, 
and then old elements are removed and new elements are introduced so that die 

3 0 mesh quality inqnoves. This iterative process continues until one is satisfied 

with the overall quality of the mesh. In one preferred embodiment, four node 
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tetrahedral elements ace used. The initial mesh is constructed by applying a 3-D 
Delaunay triangulation on the starting set of pomts. The iterative process 
identifies poorly sh^>ed elements using an element quality measure, and 
introduces new points and remeshes the region. The process terminates when all 
5 elements meet a minimum quality criteria. The preferred implementation uses 

longest edge bisection to introduce new points that improve the mesh, but other 
methods can be used. 

The fifth step processes the constraints described in step one into a 
system of linear constraints. In the preferred embodiment, the final system of 
1 0 constraints is linear in terms of the nodal displacements at the vertices of the 

tetrahedral elements. The desired form of the constraints is: 

Cu = q (6) 
The matrix C contains constant coefficients. The number of rows of C is 
equal to the number of constraints in the system. The vector u represents the 3- 
D displacements of the vertices of die tetrahedral elements. The vector q 
1 5 contains constant coefficients. If the constraints are homogenous then each 

element of q will be 0. The foiin of constraint specification given in Equation 
(6) allows arbitrary linear multipoint (involving more than one vertex) 
constraints. 

The conversion of the constraints specified in step one into the form 
20 shown above depends on the type of constraints involved. For two points to be 

tied together the constraint would be: 

p,+u,=P3+u, (7) 
or u, -Uj =P2 -p, (8) 

In these equations, p, and are vectors finm the origin to the vertices 
of interest, while u, and n, are the displacements of the same vertices during 
warping. Equation (7) denuuids that the final location of each vertex, equal to 
25 the starting point plus the displacement during warping, be equal to tiie final 

location of the otiier pomt Equation (8) is in the form of Equation (6), with 
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q - p, -Pi and results in three linear constraints, in terms of the x, y, and z 
components of the nodal displacements. Expanding Equation (8) into tiberee 
equations in the form of Equation (6) then gives: 



10 0-1 
0 10 0 
0 0 10 



0 0 
-1 0 
0 -1 



(9) 



Other constraints, like the distance between two points, are non-linear in 
nature. The nonlinear constraints can use the existing geometry of the system as 
well as small deformation assumptions to produce linear multipoint constnitils. 
For example, to specify the dented distance between two points to be stmt 
specified value x, one could determine the vector v,, between the final pmiHi 
locations: 

v„=(p,+n,)-(p,+u,) (10) 
and then specify the desired length of the vector: 

h.l=' (11) 

or, using the vector dot product: 
v««v„-x» (12) 

Both Equations (1 1) and (12) are nonlinear in terms of the displacements 
of the nodes, u, and Uj . To linearize the constraint we can specify the desired 
length along die original line of action to be equal to the desired ofibet: 



Ipj -Pil 



W [0*2 +«j)-(P, +■,)]•■« = 
«I •■21 -■!•»« =JC-|P2-P|| 



(13) 

(14) 
(15) 
(16) 
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The term on the right hand side of equation (16) is the desired distance 
between the points minus the current distance between the points. The x, y and z 
components of nj, are constraints. Equation (16) can be written as a single 
constraint in the ]xoper form as: 



{-"ii* ""21^ -"lu "21* 'hm "11.)^ 



-Hp. -P.I (17) 



In Step 6 the final system of linear equations is assembled. There are 
two parts to this step: first, assembling the element stiffnesses for each of the 
tetrahedral elements, and second, selecting and applying a constraint handling 
technique. The calculation and assembly of element stiffiiesses follow standard 
finite element procedures. Using constraints in the form of Equation (6) 
involves a constraint processing method. The Lagrange Multipliers technique 
can be used to introduce the effect of the linear constraints, but any other 
method, such as penalty or transformation techniques, coidd be used equally 
effectively. 

Using Lagrange Multipliers, one introduces a new variable into the final 
system of equations for each constraint in the system. One then modifies the 
static equilibrium equations for the unconstrained system, which arc given by: 



Kusr 



(18) 



In Equation (18), K is the system stiffness nutrix, assembled from the 
individual element stiffiwss contributions, u is the displacement vector that is 
the solution of the problem, and r is a vector of externally iqiplied loads. In this 
embodiment of the invention, there are no externally applied loads, so the r 
vector contains only zeroes. Equation (18) does not include the effect of any 
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constraints, Init these can be included using the Lagnmge Multiplieis tedinique 
to give the system of equations: 



In Equation (19) K, C, u, r, and q are as previously defined, and is a 
vector containing the additional Lagrange Muhiplier variables that are ^ 
introduced using this method. The matrix is the transpose of C, and Qis « 
matrix of zeroes. The solution of Equation (19) gives the displacements u that 
satisfy the linear constraints described by C and q. Note that these constraints 
may be linearizations of nonlinear constraints, and the nonlinear constraints 
might not be satisfied by the solution at this point. 

If penalty or transformation methods were used instead of Lagrange 
Multipliers, a system of linear equations different from those shown in Equation 
(19) would be produced, but the solution of the linear system of equations will 
give similar values for the displacement vector u. 

In Step 7 Equation (19) is solved to give u and . There are many 
methods available to solve large systems of linear equations, and the preferred 
embodiment uses a symmetric solver with a profile storage scheme. The 
dififerem types of solvers that could be used give essentially the same results, 
but optimize speed and memory usage differently. 

The preferred embodinwnt uses a direct solver, but iterative sparse 
solvers could be used as weU. The system of equations shown in Equation (19) 
is sparse, so significant speed enhancements can be achieved by selecting the 
proper solver. However, the results of the waiping piocess overall are 
unaffected by this choice. 

In Step 8. one must check if the cunent displacement satisfies the 
constraints to a desired level of accuracy. If the current deformed shape violates 
the offset or angle in any of the constraints collected in Step 1 by more than a 
user specified tolerance, then steps 5 through 7 must be lepeated, starting with 




(19) 
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the new defonned shape. The linearizations of the dupe may change on each 
iteration since the geometry of the volume changes with the cumulative 
defoimations. When all constraints are satisfied within the given tolerance, then 
one can proceed to step 9. 

Step 9 uses the nodal deformations u calculated in Step 7 to determine 
the deformation of any point of interest within the volumes. For each point of 
interest, one must find an finite element that includes the point on its surface or 
interior. If the point is internal to an element then only one such element exists. 
If the point is on the surface of an element or along the edge of an element, then 
several elements could be considered to contain the point. Any of these 
elements can be selected to determine where the point of interest moves. If the 
point is shared between elements, then the use of any of the elements to find the 
point displacement will give the same results. Once an element is identified, the 
vertex displacements of that element are extracted from u and are used to 
determine the displacement of any point on the interior using an interpolation 
process. This procedure uses the finite element shiqx functions which are linear 
in the preferred embodiment, and is a common operation in finite element 
aiulysis. 

Auto-segmentation 

The novel auto-segmentation process, as presented below, involves a 
similar sequence of operations to the manual modeling process described 
previously. A point cloud is segmented, geometric primitive objects are fit to 
the point groups, and then modeling operations, such as extension and 
intersection are used to complete the model. In this novel process, automation is 
applied to each of these stq)s, as well as the entire process, as described below. 

It is possible to automatically partition the scan points into groups 
representing primitive geometrical shapes by using variations of common 
machine vision techniques. A gridded scan field is stored in a two dimensional 
array of points, much like a regular bitmap. The scan field differs fmm a bitmap 
in that more information is stored at each location than just a color. Each point 
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stores its loctrtion in space, from \^ch the distance to the scanner can be 
caloilated, as well as the intensity of the return laser pulse. The dqrth 
mfonnation calculated fit>m the tfuee dimensional position stored at line points 
is crucial to Ae automated segmentation algorithm described here, even fhOi^ 
many operations, such as filtering, rating, thresholding and thinning are 
commonly used image manipulation operations. 

The first stage of the auto-segmentation process is to estimate the 
sur&ce normal at each point in the grid. This can be achieved using many 
different techniques, the current embodiment of the software fits a plane to tiie 
nearest neighbors of the point in Ae 3 by 3 grid surrounding it The noRn4i|r 
the resulting plane is taken as the normal at the center point Each point iii tie 
grid has a normal calculated in the same way, except that edge and comer foints 
ignore the missing neighbors in the normal calculation. The normal stored 
each point is a tiuee dintensional vector and is noimalized to have unit lengUL 

In tiie seomd phase two ratmg images are created by convolving 
standard edge detection filters over the grid. The first rating image is creaied by 
convolving the depth of the grid point witft an edge detection filter to ideitfify 
depdi discontinuities, such as those that would occur at an occluded edge. A 
variety of edge detection filters can be used, but rather than operate on color or 
intaisity the filter operates on the depth information stored at each grid pomt 

The second rating image is created by convolving the normal with an 
edge detection filter. The normal rating image is actually composed of 3 
subimages created from a cmivolution with die normal's x, y, and z 
components. The resulting three values are combined by taking the square root 
of the sum of tfie squares to give a per-point scalar value. The secoiul mtionf 
image is used to identify normal discontinuities, such as those that would occur 
at the edge between a wall and a floor. Again, a wide variety of edge detection 
filters can be used, but the values used are normal coefficients rather than color 
or intensity. 

Once the two rating images have been created they must separately be 
converted to binary images. Conventional machuie vision algtmdims, sudh as 
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recursive thresholding can be used to achieve this task. Each point in the depth 
and normal rating inures contains an estimate of the gradient of the depth and 
normal respectively. Recursive thresholding can be used to isolate the regions of 
highest gradient In the resulting binary images the points in the regions of 
5 highest gradient are marked as edge points while the rest of the points are 

marked as non-edge. 

A final binary image is created by marking a point as an edge point if it 
is marked as an edge point in either or both of the two binary images created by 
recursive thresholding above. All other points are marked as non-edge. This 

1 0 image contains all edge points that delineate the boundaries between groups of 

points on different surfaces. 

The final step of the point partitioning process is to use a connected 
components algorithm to collect the points into groups separated by edges. 
Points are considered to be connected only if they are vertically or horizontally 

15 adjacem in the grid, diagoiud adjacency is not used. Very simple algorithms can 

be used to identify the unique groups of non-edge points in the image. Each 
group of connected points is then cut from the initial point set to form a new 
group of points. The result of this algorithm is the partitioning of the point set 
into multiple point groups that each represents a single surface. Each of the new 

2 0 point groups can be fit by a geometric primitive as described in the next section. 

Once the scan cloud has been partitioned into groups of scan points that 
lie on different surfaces, the next step is to fit objects to the desired surfaces. A 
variety of methods can be used to achieve tfiis task. The current embodiment of 
the software can perform the object fitting process in two different ways. The 

2 5 first method fits a series of objects to each group of points, and selects the 

objects that produces the smallest distance enors between the measured points 
and the fitted object surfaces. The second method uses the quadric sur&ce fit 
described previously, and resulting principle curvatures, to determine if a plane, 
cylinder or sphere should be fit to a particular point group. Other variations of 

3 0 these ^proaches could also be used, such as progressive commitment, where 

objects are fitted in order from simplest to most complicated, and the process 
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stc^s whenever the errors associated with the particular fit drop to acceptsMe 
levels. 

The last stage of auto-segmentation process extends primitive objects, 
vAvac possible, to create complete object intersections, rather than stopping at 
scan point boundaries. Using the gridded nature of the original data and the 
edge mformation from the point partitioning algorithm described above, it is 
possible to extend and intersect objects. For all edges that result from surface 
intersections, which are the sur&ce nonnal discontinuity edges described ^borve. 
one can extend the objects on either side of the edge to form an tntersectloi^ 

Model Annotation 

In order to compose a semantically rich 3-D model, individual parts m 
the above geometrical model can be annotated with additional, possibly mm- 
geometrie, information, sudi as material references <x part numbers. Tdiis 
information can be entered manually through a special window for displaying 
object attributes. 

The user nuy click on an individual part in the geometrical model and 
recover such additional infonnation through other windows. Similarly, the ma 
may request that all parts which meet some selection criteria are to be 
highlighted. 

A novel method is also used for automatic model annotation. This 
method uses the FDV 10 to scan bar codes containing pertinent infonnation 
relating to any given object Standard bar code reading and decoding teclmk|iKs 
are used to convert optical information to useful digital information that is 
associated with a given object scanned at the same time as the bar code. The 
captured information can be displayed as described above for the manual 
method. 

Geometry Display and Queiy 

The model is accessible in a variety of ways, including access dm>ugh 
the data window 1610 where the model is rendered. Many standard graphic 
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interface techniques can be used to manipulate the view of the model; in one 
preferred embodiment, a crystal ball inter&ce is used. Any object in view can be 
selected using the pointing device; its geometrical attributes can then be 
displayed, in addition to other annotations added to the object. Traversing the 
5 data set can be simplified by placing objects in different layers, and then 

displaying only the layers of interest. Reducing the number of rendered objects 
in this way increases the interactive performance of the program. In addition to 
querying the geometric properties of individual objects several standard tools 
for measuring distances and angles between objects can be employed. 
1 0 Additional standard techniques can be used for operations such as contour 

generation, 2-D section cutting, and automated dimensioning. 

The resulting model can be exported to any of a number of CAD 
programs for further editing or designing. In the preferred embodiment, the 
CGP 40 can create a CAD file in a format compatible with several commercial 
15 CAD programs, and can then start the CAD program, having it load the 

generated data file. With both the CGP and the CAD program running, the user 
can then view and work with the model using either program. 

The following documents form an integral part of this specification: 
Modular Decomposition 
2 0 Sunmiary of Proposed CGP Specification 

Cyrax Software Specification 
Product Overview 
Collection of View Slides 
Introduction and Overview 
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Modular Decomposition 



CQIIFIDENIIAL 



The user interface module encapsulates all parts of the code which handle the direct 
interaction of the user with the functionality of the systenL The functionality of the 
software will be implemented separately ftom the user interface to diat funcdmality. 
This will facilitate the possible implementation of an API and insure that die -"-^ 
application is not bound to a particular style of user interface. 

This module consists of such items as: " ^ 

- wndow management 

- menu* tool bar, and scene viewer creation 

- event dispatching 

- user-intarface look (colors, icon placement, etc.) 

The user interface module will interact closely widi the model viewing module .^hich 
manages display of die object database. The user interface module will not hionik 
display of, picldng in, or management of scene graphs. This functionality wiU be 
encapsulated in die Model Viewing Module. 

This module will also tely heavily on the Finite State Machine module which provides 
die basic tools necessary to control die changing state and feedback of the user . 
interface and for determining which functions of the system are available u> dtt user at 
any given time. 

MnHriVUwin.^ Module 

The model viewing module encapsulates all functionality involving the displi^ of the 
object database and display of feedback and auxiliary information in the objeCT Scene. 
Most of die base functionality is provided by Inventor but a strong encapsulation of 
diis module will prevent dependaoce on Inventor. 

Picking of objects in the scene should be handled by a separate module (which might, 
in a simplified implementation, make calls to the picking functionality of diis xnodule) 
to permit the implementation of fast picking mechanisms using a spatial dawhlW!. 

This module consist of such items as: 

-> visual object node constroction from geometric objects 

- scene rendering 

- setting rendering options (lighting, shading, texture, etc) 

- change of viewpoint 

- highlighting/feedback operations 

The model viewing module interacts closely with the User Interface Module to provide 
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interactive viewing. Because of the interactive viewing functionality provided by 
Inventor viewer, the separation of interactive view manipulation (typically pan ot the 
User Interface Module) from scene management may be difficult or possibly 
undesirable. The User Interface Module will rely on funcuonality provided by this 
module to provide feedback within the scene to indicate operands to tools, selected 
objects, etc. 

The model viewing module is tighUy coupled to the Database Module and the 
geometric objects which it contains. Geometric objects will be stored separately in the 
database and the scene requiring that care be taken to insure the consistency of the 
object database with the scene diat the user is viewing. This separation allows such 
operations as viewing subsets of the database, highUghting an object as feedback to a 
tool, and use of spatial hierarchies within the database to be performed intuitively. 

Finiti. Staf f Marh'"^ MnHnlc 

The finite state machine module encapsulates the notion of an extensible system for 
soecifving a collection of states, actions performed in those states, permissible 
transitions between states, and correspondences betw«sn events f ^ ttansijions^^^^^ 
User Interface Module will make heavy use of the FSM module to determine what 
tools and objects are available for selection at any time. 

The finite state machine module consists of items such as: 

- Event handling 

- Marking tools as inaccessible (grayed) 

- Setting proper state after an error 

- Calling tool implementations 

- Handling operand indication 

The finite state machine module is difficuh because of the need for exact specification 
I?Sie state o^^^^^^ interface despite its extensibiUty Thus, menu items for a ttX)l 
which may not exist when another tool is created may have to be grayed when that 
other tool is selected. 

The database module encapsulates storage and retrieval of the 
aDDlicaiion It should provide rapid access to that data, whether it is stored on disk or 
Tm^o^' in a^spkrent manner. This module should be designed to permit a 
diSe™ module in which multiple clients operate on the same database. 

A scheme for checking out objects to be used by an operation seems necessary aj well 
as ll^iSL of cachilg (assuming that disk support for the database is provided). The 
^S^e Slid provide a general structure upon which a spatial hierarchy can be 
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imposed for more efficient operation. A structural hierarchy and instancing should 
also be provided. „, 

The database module consists of such items as: 

- Addition of geometric objects and images to the database 

- Retrieval of geometric objects and images from the database 

- Disk storage and caching 

- Checking in/out of database objects for editing operations 

The database module vrill be closely coupled to much of the system. All modUtat 
which operate on geometiic objects will interface with this module. '^'f 

Tnnnf/OiitinitMndute 

The I/O module encapsulates the transition of the geometric database from a run-time 
state to an inoperative state and back. The I/O module also encapsulated the 
translation ot a geometiic database to various third party CAD formats (DXF, and 
the construction of an internal geometric database from such formats. ; 

The I/O module consists of such items as: 5£ 

- Save/load database 

- Save as... 

- Import frcrni... 

The I/O module is closely coupled to the Daubase Module and the Geometric tDbjects 
Module. 

n^nntmtrir Ohyw t Mniiule 

The geometric object module encapsulates the various geometric primitives sopponed 
by the system. The module will contain submodules for each of the primitives and will 
provide the basic functionality accessible through the various tools in the Tool Module. 
The primitives are mtended to be a fairly small, finite set of objects. In additikm, Aere 
will be a group object which is a collection of primitive objects. It is undesirable to 
have a primitive for every possible object of interest (piping elbow, ladder, etc^. 

The geometric object module will interact closely wift the Database Module tidiuch 
provider persistent storage of geometric objects and the relationships betweenlKem 
(see also the Dependency Module). The geometric objea module will also interact 
closely widi the Model Viewing Module which stores and displays a parallel visautl 
representatim of geometric objects. Most of die system functionality on geometric 
objects will be encapsulated in tools within the Tool Module so that, for eiiangpe, 
rather than adding a constructor to a line object for generating a line from two|3anes. a 
line-from-two-planes tool generates a call to a line constructor when given two 
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planes. This follows a model which supports extensibility through the addition of new 
tools rather than the alteration of existing primitive objects. 

The geometric object module consists of such items as: 

- Facet, point, edge. line, and cylinder construction 

- Generating calls to Model Viewing Module to create a display of an object 

- Changes to an object such as position and orientation, color, etc. 

- Queries to geometry of an object such as bounding box 

Dfnf nrif nc y Module 

The dependency module encapsulates the notion of a historical genesis of the 
geometric database. Objects are generated from other objects, and the system records 
this generative process so that it can be repeated automatically when changes or 
additions are made to ancestors of a given piece of data. 

This module interacts closely with and will have its dau stored in the Database 
Module. The depency information is a part of the geometric module and must be 
stored persistently. A strong encapsulation of this dependency information is desired 
so that geometric primitives need not maintain any information about their genealogy. 

The dependency module consists of such items as: 

- Recording the operations and operands required to generate a geometric object 

- Regenerating all of the descendants of a changed objects 

TTnHn Module 

The undo module encapsulates the run time history of a user with a geometric 
database. The undo module records a stack of actions necessary to undo operations in 
the reverse order in which they were perfoimed by a user. Each undo-able tool in the 
Tool Module will provide a call with operands which when run will reverse the acuons 
of that tool. 

The undo itaodule provides functionality to: 
-Store the 'undo* calls and operands 

- Apply these 'undo* calls when requested and generate a redo stack 

The undo module interacts closely with the Tool Module as each undo-able tool must 
provide appropriate calls to the undo module. The undo module, while interacting 
closely with the Database Module, does not stores its data in the database module 
because it records a session with a user rather than information required in the 
geometric model. 
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The tool manager module encapsulates providing extensible functionality available to the 
user. The tool module is the key participant in the extensibility of the system. The addition 
of new functionality to the system will coincide with the addition of a new tool. 

Tools are expected to provide a default method of invocation, a method of selecting 
arguments, an extension to the user interface, an extension to the help system, and an 
implementaticHi of the functionality of the tool. Tools must instruct the User Interface 
Module and the Model Viewing Module to provide appropriate feedback to the user and 
must gracefully handle abortion by the user or internal errors (see the Error Module). 

The tool manager module consists of items such as: , - 

- Registering a tool with the application i 

- Invoking a tool when requested by the User Interface 

- Remapping how a tool is invoked 

The tool manager module interacts closely with the User Interface Module and the Rnite 
State Machine Module as well as the Tool Kit Module which provides the actual tools 
which the Tool Manager manages. The Finite State Machine Module allows ^ tool 
manager to instruct the User Interface Module how it is to behave as a tool is sdected or 
being used. 

Tnnl Kit Mniiiile 

The tool kit module is an extensible collection of tool objects each of which can be given to 
the Tool Manager Module for management. Tools encapsulate the functionality of the 
system provid^ to the user. Each tool provides a default method of invocation, a method of 
selecting arguments, an extension of the user interface (provided through the FSM module) 
to permit feedback to the user specific to the given tool, an extension to the hdp system, and 
an implementation of the functionality of the tool (including calls to the Database and 
Geometric Objects Modules to create, destroy, and edit geometry). 

The tool kit consists of all items of system functionality not describe elsewhere in the 
design. ; 

Error Module 

The error module encapsulates the general error handling mechanism of the system. This 
module will be used throughout the system and will allow for the provision of error 
handlers, the repotting of errors to the user (through calls to the User Interface Module), the 
addition of assertions within the code, and the signalling of errors when assertions are 
botched. This module is likely to require the addition of some basic constructs to the coding 
paradigm. 
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The error module consists of items such as: 



- Error handlers 

- Assertions 

- Signalling cirors 
-Error reporting 



The enor module will interact with all modules of the system and must be 
implemented with a focus on making its presence as minimally mtrusive to the 
programmer as possible. 



The help module encapsulates an extensible context sensitive, on-line manual. The 
helo system will provide both a page oriented general manual as well as abbreviated 
context hilpinfo^^^ ("What dS I do now?"). Each tool wUl provide an extension 
to the help system and links into that extension within its finite state machine to permit 
context sensitivity. 

The help module consists of items such as: 

- Display and navigation of help pages 

- Indexing and searching of help information 

- On-screen, interactive help messages 

The help module wiU interact closely with the Tool Kit and Tool Mamiger Modules as 
well as the User Interface Module. 

nata rnii^grinn Module 

The data collection module encapsulates the acquisition of data ^ro™. J « P^JJ^J^^^^^^ 
world Data acquisition objects such as scanners and cameras arc objects 
Suic The d?u collecUon module provides a collection of commands which cmi be 
Sfomed on a picie of data coUecti^ hardware and mechanisins for returning data 
fromSfhardwSie to the system. For an extensible daui coUecuon system daw 
SheriSg hSwaie would have to provide a user-mterfacc extension to set parameters 
for thSdware or command that hardware to perform data gathenng tasks. 

The data collection module consists of items such as: 

- Communication with cameras and scanners .... .1,- H*tiiri^ 
_ A iSw interface for generating commands and receiving data from the devices 

- Calls to system functionality for adding data to the database 

The data collection module interacts closely with the Database Module and the User 
Interface Module. 



PCT/DS97W793 



The registration module encapsulates the process of registerin; and combining two 
geometric databases. Tlie registration module includes a xar- jod for selecting two 
geometric databases, a method for specifying conespondetices between databases, and 
a method for constructing a merged geometric database from the two sepaiattt - 
geometric databases. 

The registration module intetacts closely with the Database Module and the User 
Interface Module. 

The layers module encapsulates the presence of named sets of objects within a 
geometric database and a collection of tools which permits the selective display of and 
operation on such sets. The contents of this module are currently ill-defmed. 

The layers module would be expected to interact closely with the Database Alpule and 
the Model Viewing Module. 

Ahfitract Date Typfs Module 

The abstract data types module encapsulates a collection of general purpose abstract 
data types which are separable from the data types specific to this application. ^ 

The abstract data types module consists of items such as: 

- Stack 
-Queue 

- Single and double linked lists 
-Hash table 

The abstract data type module will be used in virtually every other module. 
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DRAFT 

(Confidential) 

Summary of Proposed CGP Specifications 
Smart Element Capabilities 

1. Geometry is represented initially as a cloud of points; later as a mesh, as a detailed 
model copied from a library or catalog, or as simplified model generated 
interactively. 

2. This geometry can be enhanced with color, intensity, texture, or even video 
information which gets added to the individual surfaces of the element. 

3. There may be more than one representation for each element: First there may be 
several geometrical proxies of various levels of simplification or abstractions which 
can be chosen to make rendering more efficient and the produced views less 
cluttered. 

Funherroore there can be symbolic representations for schematic renderings such as 

floorplans. elevations, or isometric pipe drawings. 

There may even be sman models sutuble for certain types of simulations. 

4. Elements can be provided with tags or "layer" information for classification 
purposes. When constructing a model of a plant or of a refinery, the user may choose 
tags from previously defined hierarchical menus; e.g., the choices at a first level 
might be: 

• Supporting Structure Maintenance Gangways - Pipe Systems - Other ... 
And at a second level within the category "Pipe Systems" we might fmd: 

• Steam - Water - Gas - Oil -- Alcohol ... 

5. Arbitrary textual information can be attached to each element via a text window. This 
may contain summary information firom a catalog or notes concerning the modeling. 

6. Eiemeius can be linked to other information contained in external databases, such as 
the type of a valve or pump, its ratings, its manufacturer, its maintenance schedule ... 
This information in the external database may be in the form of a hyperiinked 
database. 

It is viewed and browsed in the context of the viewer for that database. 

7. It might be useful to have some explicit connectivity information. While logical 
connectivity between pipes or electrical wiring may be outside the initial charter of 
(he CGP and could be extracted later from the implicit geometrical near-coincidences 
of certain elements, it might tie worthwhile saving such adjacency information when 
it is first discovered. E.g., when a connected mesh is decomposed into several 
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elements, say a pump and a pipe and a valve, then the cut lines in the initial mesh 
could be used to establish some explicit contact infonnation between the pans that 
share these cut lines. This information will make it later much more efficient to find 
and extract a run of connected pipe elements. 



Functionality of the Overall System 

Management Tools 

1 . At the "OS^-level, the system provides the capabilities to control the scan 
management and the video gathering. Later we may provide a user interface to 
provide more selective ways of indicating the areas that need to be scanned: one 
option is the 'nfakk-brush" paradigm in which the user outlines the features that 
shouM be incorporated in the model; die system then goes and fmds the 
corresponding fieaoues in the world and automatically selects a suiuble scan region 
for the chosen edge, pipe, or I-beam. 

2. Also at the OS level the sysam provides the capabilities for efficient muhi-tasking, 
so that several windows can be opened simultaneously, providing views into one or 
more catalogs or da t abases, while a scanning process or a modeling operation are 
underway. 

3. There wUl be data management facUities for the scanned point sets and for any 
coUeaed video images: as weU as ways to label them and file them away fw archival 
purposes. 



McKfeling Tools 

4. At the "purs geometry" level, basic tools permit selection of subsets of the scanned 
points and enable manual segmentation. Meshes can be fit over the scanned pc^nts. 
and heuristically disconnected at possible horizon lines where there ate sudden 
dramatic changes in the depth coordinates. If necessary', these meshes can be 
simplified. Planar polygons can be fit to selected portions of these meshes or to 
subsets of the original scanned poims. Survey points can be established; lines can be 
drawn between them; and break-lines can be established between polygons or 
portions of a mesh. 

5. At the "discrete elemem" level, tooU permit collections of pure geometrv parts to be 
lumped together and named as an element which can then be provided with additional 
informatioo. Sections of a mesh could be replaced with models from a catalog or 
library or with a symbolic geometrical proxy. Backsides of such models may have to 
be intelligently added, out-lier points removed, and the model generally cleaned-up 
and subjected to some consistency tesu. 

6. These dements can be provided with color, intensity, texture, or texture map 
information. Other attributes can be assigned to these objects, and various kind of 
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linkages can be established. 

7. Sets of objects can be selected interactively and grouped hierarchically. 

8. View integration is provided between views taken from different camera positions: 

• Pairwise view alignment based on registration features or based on an 
optimization process that tries to register the overlapping ponions of two 
meshes by minimizing their mumal distances. 

• Data merging by weighted averaging of position parameters of discrete 
elements or by ziiqiering together two overlapping mesh sections. 

• Est^lishing global closure by smanly distributing the closure error over 
previously established pairwise view registrations. 

^This is a hard problem that needs to be thought about carefully>. 



Visualization 

9. At any time during the data gathering or modeling process, the existing geometrical 
data can be viewed interactively either in a "crystal-ball / model-in-hand" paradigm 
or in a "walk-through" mode. At a later time we will add the proper spatial data 
structures and management schemes to make walk-through possible at interactive 
speeds even if the model is very large (more than 1.000.000 polygons). 

10. Different modes of rendering the data will be provided: point clouds - wire frame ~ 
hidden lines removed, showing unshaded but oudined visible faces - flat-shaded 
facets ~ Gouraud-smooth-shaded meshes. 

1 1 . There will be schematic and symbolic representations such as floorplans and 
elevations which may use 2D proxies for various components to make the result more 
efficient and more effecuvc. There may be center-line representations for pipes, or 
symbolic moment frame grids for steel structures. There may also be Pipe-ISO 
drawings and perhaps P&ID process drawings. 

Database Queries 

12. Geometrical information can be extracted from the noodel at any time. Cut sections 
~ can be created, and distances between two points or angles between to planes can be 

extracted. 

13. By selecung discrete elements in a particular manner (e.g.. double clicking it) the 
attribute information associated with that elements can be fetched from the database. 
If this information is large and organized in a hyperlinked document fonmat, dien the 
corresponding database browser wiU be opened and the coot page fbr this object will 
be presented. 
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Data Exchange with Other Systems 

14. Data generated within the CYRAX CGP can be exported in an ASCII text foanat in 
the form of ".crx" files. The xtx fonnat should be able to encode aU infortna^ODi that 
is self-contained within a CYRAX model. This should allow sophisticated users to 
build external programs that can search CYRAX models for specialized analyses 
(such as counting all elements of a particular type) or to make global modificatiras to 
such a taodcl (e.g.. replace all I>beams of a particular type) without the need 19 go 
through any other database system. 

15. We will provide some converters for popular existing databases such as Ant^U) or 
MicroStation which will fnoperly insert the information contained in the .eiil ittft 
stream into these databases. Ccmversely we will also provide some conveitniiiat 
will extract geometric information relevam to the CYRAX CGP and bring i^po our 
own modeling environment • r^'l^f 

16. Initially we do not plan to provide a complete smart product model database in which 
all additional information associated with any discrete model element can be stored: 
we will rely on the users of CYRAX to use their own established dau storage 
facilities. We will, however, provide facilities to gain access to such databases via a 
separate window opened at the OS level. Links between non-CYRAX data items can 
then be established directly within these external databases between the geoR»tricai 
representation of a model pan and any other entries in the database that seem 
relevant. 



User Interface Paradigms 

17. We plan to provide a nice window-based user interface that eomlues (at SW^^ 
seems reasonable) the established interaction culture in MicroStation and AutoCAD. 
This implies the use of icon-palettes, pull-down menus, as well as keyboard stKMtcuts. 

18. In the domain of graphical editing we will provide '*Sn^Dragging" behavlM'to 
combine precision and speed in a an cfGcient and user-fnendly manner. 

19. For some application domains we will develop specific "Object Associations" that 
will provide behavioral heuristics that will fwinimiw the number of mouse clicks a 
user has to naake to accomplish a particular objective or the number of degrees of 
freedom the user has to control explicitly in any modeling operation. We «qpect that 
the hitler embolic levels of lepresemation of a smart product modd win 
interesting challenges how the entry of complex informatim can be made ai^mple 
as possible. "'''flp''' 
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Cyrax Software Specificatfoir 

Propoaed Software F»«tura» 



CONFIOENIIAL 

I la I I 0 ( la I IIA I MB ( IIIA | IIIB |Fut./Opt 



Taraet Acauisition 


















raw video 


















corrected video 


















laser paint 

Taraet Selection (see Basic Interface/Selection) 


















Selection Parameters 


















samples in x and y 


















adaptive scannino 


















Data Acauisition 


















Axisalianed scans 


















spiral scans 


















edqe aliqned scans 


















poVaons 


















intensity data 


















t»re-siahted video color/texture 


















Positionino Funciianalitv 


















Get Point 


















Reflector Taraet 


















Scanninq simulator with error model 



















Ooen/Close/Save 














1 


raw scan data 


















plain ASCII 


















oeometric data 


















OXP (oeneral CAD 'default standard ) 


>/ 
















INV (SGI Inventor) 


















OQN (MIcroStation native format) 


















OWG (AutoCad native format) 
















? 


IGES 5 (int'l mechanical CAD standard) 










il 








deHmited ASCII 








V 










STL (stereollthooraohy machine format) 


















VDA-FS (automotive CAD) 
















? 


CADDS 5 (Comoutervision format) 
















? 


smart model data 


















SDMF (Ouid(Draw3D object standard) 


















VRML 2.0 (World Wide Web standard) 


















CRX data exchanqe (Cvra propr. format) 










k/ 








OBJ (Wavefront native format) 










^/ 








Print/Plot basic (window dumo) 


















advanced (scaiina. labels. ...) 












ll 







PCT/US97/06793 



SGI IRIX suoor 



__Windgjijfli_NT su»»iX>it 



OWect database enohie 



96/7 auBDort 

htacCoglandOSnjBgorL. 



IS 



Multitasking (OS leveft 
0LE2terCApy 



PPES/STg comoMancB 



Window Paradom ffotow 
' inic tool palettes 
Functten Kev shoi 




wo 97/40342 



-82- 



PCT/US97/06793 



Ma^es Basic (trianaulation) ■ "tshlna 


















"Shrink'MfrappinQ" (Oi ^osd dotli) 






















y 














Mssh optimization/siiTipHficatfon 










V 








Break Unas 








V 










Mesti to saometrtc oriniitiva connecttons 












>/ 






Best fit Primitives 


















line 


>/ 
















Diane (convex face) 


V 
















corner 


V 
















anoiebeain 


















box beam (constrained frustum) 


















frustum 


















I'beam 


















ooncaveface 


















cone 








>/ 










cylinder 


V 
















fianae 


















eibow 


















sphere 


















NURBS tine 


















NURBS surface 
















>/ 


Auto seomentation _ 


















plane 


















cylinder 


















Voxel 
















V 


Multiple Objects 


















intersect basic 


















with pipe elbow mesh 




V 














extend dick first then second 


















drao first to second 






V 












merge faces/lines 


















obiects 












V 






View Intearation 


















basic (3 points, line -•■ point. 3 planes. ...) 


















advanced (heuristic distributton of error) 








T 










view closure in 3 soaee (oiobai optimization) 



















Group obiects . bv hierarchy 


















bv laver 


















bv instance 










V "■ 








Labels 


















Abstractions 


















geometric (for waMhrouahs. at al) 


















symbolic (for isos. sectksns. et al) 


















CatakKi 


















manually substitute part for point cloud 








>/ 










semiautomated part preselection 












>/ ■ 






advanced catatoo browser interface 


















Connectivity 


















Bar Code reading 
















s 


interface to 20 drawino data extractton 














IS 



wo 97/40342 



PCTAJSS»M793 



Basto Views 
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Jan 96 'Ruof and "Chavfen' prDpoaais 

late Jan 96 system fiinetionaiity discussion of BK. CS. ACA 
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Product Overview 

Cyra Technologies* Field Digital Vision fFDV) unit provides ponable. rapid, and 
accurate laser range scanning technology in the field. The FDV, like a 
three-dimensional camera, is pointed at a target which it then scans to generate a 
ranse image. A range image is like a digital photographic image except that each 
pixel of the image is a point in space rather than a colored two-dimensional 
point. The FDV permits a user to rapidly gather three-dimensional dau about an 
object by gathering thousands of points from that object. 

Such data is the starting point for a multitude of modelling, design, and 
documeniaiion tasks. The Computer Graphics Perception (CGP) software kit 
provides the functionality necessary to convert collections of range images into 
three-dimensional, intelligent models. These models can be generated by 
suH'eyors as the FDV generates the range images or can be generated in the 
office by CAD operators from gathered data. Because of the wide range of 
modelling tasks, requirements, and paradigms within the various 
three-dimensional modelling communities, the CGP is designed to support a 
general framework which can be integrated easily with standard CAD tools, 
facility management software, and intelligent modelling environments. 

The funaionality of the CGP takes you from range image to complete, annouted. 
intelligent models. The structure, detail, and content of the resulting model is 
under the complete control of the user allowing the generation of ten minute 
rapid visualization models to multi-layer, intelligent facility documenution 
models. The various forms of modelling available integrate easily and 
consistently to permit highly deuiled and annotated submodels of interest to 
coexist with low resolution, automatically generated shrink-wrap models of 
context' structures intended only to aid in visualization. 

Framework 

The CGP is structured around a daubase (figure 1 ). The data stored in the 
database can come fmm a vanety sources including raw range scans and video 
images seneraied by the FDV, geometric entities generated by the CGP 
modelling process, geometric entities and associated product infonnation 
imported from caulogues. and annoution data such as facility maintenance 
information. Data within the daubase is classified as either 'internal' or 
'external' data depending upon whether the content of the data is meaning^l to 
the CGP svstem. Internal dau such as geometry, scan points, and geometric 
unnouiion's is used directly by the CGP is general purpose dau required to meet 
the modfellins needs of a wide range of user communides. External dau stored in 
the daubase has meaning only in the context of general database queries or 
external, application specific software. Examples of external data include the 
date a pump in a chemical plant was serviced, the manufacturer of a particular 
part, or a pointer to the home page of the designer of an aichiiecniral subsystem. 
White internal data can reference external data elements, the CGP has no way to 
understand, manipulate, or present this data aiui mustjely.Qn £xtenia).^.ries and 
eilierhal packaces to make iise of it: A more complete description of this concept 
is presented in'ihc section "Intelligent Modelling". 
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Fig. 1 - Framework 



The process of con>-ening range dau inio three-dimensional models. t\i)icailv 
termed reality acquisiuon. appears on the left side of the daubase in Fi'sure i To 
The nght of the database can be found two tjpes of tasks: those involving the 
presenuuon of the data to a user community (which can range from a sincle 
person to every deparanem in a chemical plant) and those involving the import 
and expon of geomeaic ^ from and to other sources. These tasks are not 
specific to the use of the FDV and do not inralve eeometr\' constnicnon and 
manipulation by the user. 

The realit}- acquisition process subdivides into two processes: scanning and 
modelling. The CCP provides ail of the necessarv controls to operate the FDV 
scanner to gather video images and range scans both of which arc archivea in the 
database. Conffoi over the amount of dtaa sathered. the resolution of that data, 
wd the exact region to be scanned is provided to the user thioush the Scan 
Control tools. ~ -SiiS:' 

Three toolkits are provided by the CCP to develop range scans into geometric 
models depending upon the type of model desired and the contents of the scene 
being rnodeiled. Each toolkit is optimized fw a panicular stvle of geometric 
modd mg and thus the use of different toolkits for differem iwnions of the 
modellmg task is expected. The Visual Magic toolkit provides rapid mesh 
generaoon funcnonaJity for general purpose model construction of complex 
obi«B or scenes. The Prefab Lab provides roechanisiu for replacins ranee and 
mesh data with prefabncated pans whose models are input from external ' 
catalogues or are instannauons of objects eenerated usin£ the CGP. The 
Structure Shop provides point, line, and plane primitivesand various Euclidean 
operauons on these primitives useftii for the construction of basic siruciutal 
geometi^' such as architectural structures. As wilt be described in depth in the 
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section on "Object Types", provisions for interoperabiiity between the three 
toolkits permit users of the system to select which ever toolkit seems appropriate 
For a given task and then to integrate the results of the different toolkits into a 
single coherent model. 

The Inquir>' and Presentation system is a visual front-end to the database which 
allows users to view, query, and augment die external data within the database. 
The user will have the ability to walk through generated models and subsystems, 
visualize portions of the database, or generate baste structural diagrams. External 
information such as maintenance plans, subsystem names, and references to other 
databases can be associated with objects in the CGP database. This data can be 
queried, retrieved, and updated by a user community throughout the lifecycle of 
the real world system which the database represents. 

The Data Exchange system petrnits the CGP to be coupled to existing CAD 
systems. This coupling comes in die form of dau conversion routines to and 
from various CAD formats. Vendor pan catalogues which have not been 
generated using the FDV and CGP can be imported into the system for use in the 
Prefab Lab. A part which needs to be redesigned can be exported from the CGP 
database to AutoCAD or other CAD packages to be reworked and then imponed 
bark into the CGP database. 

The various systems and toolkiu of Figure 1 are tightly integrated into a single 
coherent system from the perspective of die user. The user operates in the visual 
window-icon-mouse-and-pointer environment and moves from module to 
module simply by pulling down a different menu or accessing a different tool 
bar. Most operations do not require keyboard input or arcane command line 
arguments typical in many CAD systems: rather, the user models geometr>' by 
selecting commands from a small but powerful tool set and clicking on the 
objects upon which he wishes these commands to operate. Visual interactivity is 
key to the field usability of the svstem and thus a focal point in the development 
of the CGP. 

Document Overview 

The remainder of this document is a high level specification of the CGP in the 
form of a ussr-oriented functionality guide to the system. This format is concise 
yet descriptive but should not be mistaken for a complete requirements 
specification (which would conuin performance details, error handling issues, 
and other details which would be premature at the time of this writing). 
Similarly, although many functions will be described in terms of what the user 
will do to perform an operation ("The user clicks on a point set and then clicks on 
the planie tool to generate a plane from the set of points."), this should not be 
mistaken for a specification of a fixed user-interface. Rather, this description is 
nothing more than a simple way of clearly describing an operation (the 
Poini-Set-to-Plane operation in this case) which will be available through some 
user-interface mechanism. User swdies. user-interface prototyping, and 
ultimately a user-interface specification will precede determining the best 
manner bv which a piece of available functionality will be accessed by a user. 
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The document is divided into sections based upon the svsiem framework or 
Mgure 1 . Each subsystem and toolkit is described bv Wav of the funcnonalitv 
™S?,h,Sir.h?nf ,f '- P* " geniraies. Indicauon, ire 

made ihrougnout the document as to where expanded funcnonalitv will be aaded 
which exceeds the scope of the rlrsi fcncnuion system. These indicauons a?^ 



Scan Control 

i\ cS*'^Srrov?a„^^ 'SS™?* """S the Scan Control functionality or ibe 
the CGP. The CCP ts responsible for providing the FDV with information aixnt 
the location and resolution of the scan to be takin. The FDV uses a noveAST 

mSiSS^.^i?t "» Scan Control 

' W ^^T'^S** «n«R«iivity and point-and-click 
openiuon paradigms which underiy the usage model of ttefiill system. 

VUbo CaptUfB '"'^ 

bunon causes an RGB image of the scene from 
'^'^ ^ ^ transfored into the CCP and displaved in 
tht S^h?«' ^ '"^ ^ arniouted. and stored in 

the database at the request or the user. A video imase mav be associated with a 
set 01 range scans « see Range-Video Association b^w) 

Zoom Capture 

I^' ~ *e *»wn a standard RGB image of a 

^JSS^^y.^^"J^^ ^'''^i on the zoom capture tool and 

inSr it toi inufe taken in die direction of the point 

dSSe aS ai^Sn/l^l^u •* annotated, stored in the 
SS'vlSiS^lri'J' of range scans in the same mamier as can a 
studard video image. An Lnzoom tool permits the user to return to the standard 

Vfdoo image Browser 

fli»^J "ser to search for an image in the database by 

ntppmg through thumbnails of video images. All of the ran^ scans associated 
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with a video image can be highlighted or selected, or the user can set the viewing 
point of a model viewer to be the''position from which a video image was taken. 

The video image browser will ultimately become part of the full featured query 
interface for the smart modelling system. Queries such as "find all of the range 
scans associated with this image' or "find all of the video images taken on 
December 12. 1995" will be pan of a general purpose visual diabase que^' 
unit. The special purpose queries listed above provide a starting point in this 
direction. 

Scanning 

The .scanninc tool is available to the user whenever a video image (standard or 
zoom) is present in the targctting window. The user indicates a scan by drains 
out an axis aligned box on the video image in the targetting window. The FDV 
will generate a range scan from the region of space indicated. 

A ran ce scan is an axis aligned, approximately rectangular grid of depth 
information. Slight errors in mirror positioning in the FDV prevent an exactl> 
reciancular grid. A ranee scan has an X and a Y resolution incUcaiing the number 
of points scanned in each of these directions. The X and Y resolution inay be set 
by the user in terms of the number of points or the instance between pomts for 
some fixed distance. 

Each arid point in the range scan Is associated with a three-dimensional 
Cartesian point and an intensity value measuring the laser intensity response at 
that point. The combination of a three-dimensional Cartesian point and a laser 
intensity value defines a scan point. A range scan, by definition, is an array ot 
scan points ordered by rows and columns. A range scan may also contain 
haaced invalid points where the scanner failed to acquire an actual scan point. 

Range .scans are automatically inserted into the working database. They may be 
named and annotated manually. 

Range-Video image Association 

Tvpicallv ranee scans are automatically associated with die video image used for 
tarcettin'c that'scan. Bv default, when a range scan is taken, the associated video 
imaae Neither standard or zoom) is stored to the daubase (if it has not been stored 
previously) and an association between the video image and dte range scan is 
made. This association can be used to access a picture of a region scanned or to 
isolate the scans taken from a given location. 

Real-time Scan Display 

Scan points being acquired by the FDV may be viewed (seeModel Displ^) 
anv active data world view while the scan is taking place. These points are not 
available for anv operation other than display until the scan is complete. 
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SeanCane§aaBon 

A scan cancellation lool is avail^le while the FDV is generating a range scan. 
This feature terminates the collecunn of scan points. The incomplete ranee :>can 
taken up to this point is inserted in the oaiafraiKF 



Visual Magic 



The Visual Magic (VM) toolkit provides rapid visualization tools for con\-erting 
scanpoims into triangular mesh models of a scene with minimal user interaction. 
The focus of the operations in the VM toolkit is to provide extremely eeneral 
fuaedenality capable of modelline anv object. Unlike with the Prefab'Lab and 
the Sifuetttre Shop described below, the VM toolkit has no nodon of primiuves 
beyond that of the triangular mesh. This permits a wide vanety of robust 
operations to be penormed on a single, general purpose data types. As de«:nbed 
in the section any geometric surface in the CCP can be convened into a 
triangular mesh and operated upon using the Visual Magic tool set. 

Range Scan to Gridded Mesh 

The Range Scan to Gridded Mesh tool provides the simplest way to generate a ... 
model using the FDV. Upon acquiring a ranee scan, the user can select that scwi 
and click Range Scan to Gridded Mesh to generate a triangular mesh from rbai'^'' 
scan. The generated mesh is triangulated based upon the rccungular gndding rtT;. 
the scan. The gridded mesh is differenuated from a non-gridded mesh which ^* 
was not generated direaly from a range scan. A gridded mesh maintains 
information about the location of the scanner and the row and column ordering of 
its points. Invalid points are not included in the mesh and tnaneles whose size 
exceeds a user specified parameter are also excluded. The ranee scan is replaced 
in the database with the mesh. Meshes can be viewed in a nuinber of wavs 
including wire-frame. Gourand shaded, and as a hidden line dtawins i see Model 
Dixplay). 

GrUded Mesh SimpttnetUm - 

Gridded meshes roav be stmpliried by remox-inc triandes in reeions which arr "'' 
essentially planar. The user can select a gridded mesis and click on this toot to 
remove as many triangles as approriate given a user specified parameter for ih« 
maximum aror to be induced into the mesh. The original griilded mesh is 
replace with the simplified gridded mesh in the rfatabnse " " 

Gridded Mesh StnooUting 

An entire gridded mesh or regions widtin a itridded mesh mav be smoothed bv 
averaging the range depth of each point with its neighbors, the t\'pe of 
averaging (Gaussian, simple average, etci and the size of the neighborhood to 
averaged over is specified by the user. Smpodiing can be limited to a particular 
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region by selecting a polygonal region of the mesh prior lo selecting this tool. 
The original mesh is replaced with the smoothed mesh in the database. Multiple 
smoothing iterations can be performed on the mesh or region by repeated 
activation of this tool. 

Mesh Alignment 

Two meshes (gridded or non-gridded) which have been approximately aligned 
manually (see Freehand Rotate. Freehand Translate, and Move to Attach) can be 
aligned more precisely using the Mesh Alignment tool. The first mesh indicated 
is the anchor mesh and does not move. The second mesh is adjusted to achieve a 
greater alignment between the two meshes in die regions where the two meshes 
overlap. The original meshes are maintained in the database with the global 
transformation matrix of the second mesh altered. This step would typically 
precede a Mesh Zippering step but might also be used to register two scenes 
(determine their relative positions in world space) generated with the Structure 
Shop or Prefab Lab taken from difiierent but overlapping viewpoints. In the latter 
case, non-mesh geometric primitives would be temporarily converted to meshes 
(see Object Polymoiphism) to perform the alignment operation. 

Mesh zippering 

Two meshes (gridded or non-gridded) may be merged into a single mesh using 
the Mesh Zippering tool. The resulting mesh is always non-gridded. Mesh 
zippering sews together regions of overlap between two meshes whenever the 
distance between the overlap is less than some user specified threshold. 
Typically this operation is preceded by a mesh alignment. The two original 
meshes are replaced by the joined, non-gridded mesh. 

Cut Mesh with Mesh 

A mesh may be cut with another mesh by using die Cut Mesh with Mesh tool 
Both meshes can be either gridded or non-^dded. The first mesh selected is the 
mesh to be cut. The second mesh is the cutting mesh and will not be changed by 
the operation. The lines of intersection of the two meshes is computed and the 
cut mesh is divided along die line of intersection. Additional points are inserted 
into the cut mesh as necessary to give a smooth intersection (no gaps) with the 
cutting mesh. The cut mesh is replaced in the database widi some number of 
non-gridded meshes representing the various components into which the cut 
mesh was sliced. This operation would typically be followed by the deletion of 
some number of the cut components and would permit the clean intersecuon. for 
example, of a mesh representing a pipe at a skewed angle with a wall or other 
more complicated object. The combination of this operanon with Object 
Polymorphism (see below) is extremely powerful. 

Mesh/Mesh Mutual Cut 

This operation is identical to the Cut Mesh with Mesh operation above except 
that both meshes cut each other and are replaced by their components as 
non-gridded meshes. Thus performing this operation on the mesh of a cylinder 
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iniersecied perpendicular to its axis bv a meshed plane fFigure 2) would result in 
four separate componenu. 




Ftj. 2 - Mesh/Mesh Mutual Cut 



TrimM98h0v»i1ap 

When two meshes overlap, it may be desirable to leave one mesh intact while 
trimming the other inesh to join it seamlessly. The Trim Mesh Overlap tool 
pen'orms this (^)eraiion by cutting the trimnwd mesh at regions where overlap 
begins (within some user specified distance), discarding the overlapping ponions 
of the trimmed mesh, and adding vertices as necessary to form a seamless join to 
a static mesh. The two input meshes can be either gri'dded or non-gridded. The 
sutic mesh remains in the database unchanged. The trinuned mesh is divided 
into one or more component meshes which are always non-gridded. This tool 
combined with Object Polymorphism (described befow) peimin the replacement 
of parts of a mesh with prefabricated objects (from the Prefab Lab belowi white 
maintaining visual conneaivity of the various pans in a scene. 

- Non-GriddBd M§sh SimpHfieation/Optimfzation 

Mtsh simplipcatian of non^qridded meshes is a compuiaionallv expensive task 
being studied at a number of major research institutions, ne CCP will, m future 
generations, provide some of the funaiotudity derived from the research efforts 
inthisfiM. 
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Prefab Lab 



The Prefab Lab (PL) toolkit provides for the intesrauon of preconsmicied 
submodels into a scanned scene. These submodels can be generated within the 
PL toolkit, imported from external catalogues or CAD programs, or constructed 
from parts senerued using other toolkits. 

CottGBpts: Grouped Oblects and Prototypes 

The Prefab Lab supports two notions which play a key role in the construction 
and use of inteliignet models: the grouped object and the prototype. A grouped 
object IS simply a collecuon of geometric primitives or other grouped objects 
which act as a single entity. This is conceptually equivalent to the grouping of 
objects in most drawing (mgrams. Grouped objects exist in the ssme coordinate 
space and aa as a rigid object. Operations performed on the group are performed 
on all objects within that group. 

A prototype is a class of object which exists outside of a geometric scene for the 
primar\- purpose of insuntiaitng that object within the scene. A prototype may 
be formed from either a single geometric primitive or a grouped object A 
prototype is essentially a meia-object" which stores general information about 
all instantiations of that prototype. Instantiations of a prototype are called pan:. 
An example will help to clarify. 

A person seneratins a model of his kitchen constructs a three-dimensional mesh 
model of a chair. Using the Visual Magic toolkit he generates separate meshes 
for each of the less of die chair and the seat. To avoid scaiming each of the four 
less, he generates a model of one leg and then uses copy and past tools to place 
the object in the scene four times. At this point, he decides to change die color of 
the less of the chair to blue. If he changes the color of one leg, will ail of the legs 
chanse color? This depends upon whedier or not the legs are pans, that is. 
instantiations of a protoivpe ( in which case, they share informauon such as color) 
or simply copies of an object (in which case, there is no stored relationship 
between the less). A prototype stared m the database may not be insuuiuaied 
anvvv here within a scene. It has an existence beyond its appearance in an active 
database. .\n object, however, is entirely self-contained as an endty in a scene. 
Removing an objea from a scene removes it from the database. 

GroupAJngroup 

The Group tool takes a set of one or more objects or paru and generates a new 
gropued object which is a hierarchical collection of these input objects. The 
srouped object mav be generated from geometric pnmitives. parts, or other 
grouped objects, the entities from which a grouped object were formed are 
replaced in the database by the new grouped object. The Ungroup tool removes a 
grouped object from the database and replaces it widi instantiadons of its 
components 
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Cut/Copy/Paste 

The Cut tool allows the user to delete one or more objects or pans from a scew 
for later Paste operations. The enuty is removed from the active daabase and ~ 
stored in the Copy/Cut buffer. The Copy tool is identical to the Cut tool exeq|» 
that the original object or pan is not removed from the active database. The 
Copy/Cut buffer is a single level buffer which stores only the last enutv cut or 
copied. 

The Paste tool places the entity in the Copy/Cut buffer into the active database. 
If the entity in the Copy/Cut buffer is a parL a new pin is created in the active 
database. If the entity in the Copy/Cut buffer is an object, a duplicate of dil$ 
object is copied into the active database. 

A single Copy/Cut buffer exists across all databases during a session so that 
entities can be cut and paste from one database to another. The Copy/Cul buffer 
is essentially a one object database which can be viewed, edited, and' manipulated 
as desited (see Data Viewing). 

FmhandTranstate 

Entities can be translated interactiveiy usins the Freehand Translate tool. A 
translation manipulaur is generated around the selected entitv permttnng the 
entity to be translated interactively. The new position is reflected in the database. 

Fn^iMnd RotBtB 

Emiiies can be rotated interactively using the Freehand Route tool. A nxaiion 
manipulator is generated around the selected entity permitting the enutv to be; 
routed imeiactively. The new orientation is reflected in the daubase. 

Move to Attach 

One entity can be moved so as to oncnt properlv with another entitv using the 
Move to Attach u}ol. The user selects up to three points on the object to be 
moved and conesponding attachment paints on a fixed object.. The interfiKce 
forces the attachment points to be consistent (that is. there exists a wav to 
position tiK movable objea so that all attachments are made). Upon identifying 
these correspondences, the entity to be moved is repositioned and onenied so as 
to enforce the attachments and the updated location is reflected in the daubase. 
Now dia attachmems are temporary and cease to exist upon the completion of 
the operation. To join die objects once they have been attached, the user should 
use the Group tool. 

Attach 

The Paste Attach tool is idenucat to the Attach tool above but ii pemiis the user 
to attach an object from the Cui/Copy buffer into the acuve data world. 
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Freehand Scale 

Entities can be scaled interactively using the Freehand Scale tool. A 
scalingmanipulator is generated around the selected entity permitting the entity to 
be scaled interactively. The new scale is reflected in the database. Scaling can 
be performed in the X.Y. and Z directions of the local coordinate system of the 
entity. For objects, scaling constraints (uniform scale, uniform scale in X and Y. 
etc) can be specified via a preferences window. For a pan instantiation, scaling 
preferences are mandated by the part. 

Scale to Attach 

Scale attachment is analogous to Move to Attach except the object is scaled 
under its constraints to make the required attachments. The new scaling of the 
object is reflected in the database. 

Object To Prototype 

The Prototype tool generates a prototype from an object or primitive and stores 
that prototype in the database. The original object or primitive is removed from 
the database and replaced with an iitstantiation of the newly generated protot\'pe 
The prototype will i)ersist in the database even if all parts associated with it are 
deleted from the active database unless an explicit Pan Deletion is requested. A 
name and annotation tnay be attached to a prototype. 

Prototype Browser/Part Instantiation 

The Prototype Browser provides the ability to search through all of the 
prototypes in the database. Prototypes are cUsplayed within the browser and can 
be dragged from the browser into an active dau world. The Move to Anach and 
Scale to Attach can also be used to generate a pan from a prototype attached to 
objects in an active data world. New parts generated from the prototype browser 
are stored in the active database with the associated placement information. 

Part to Object 

The Fan to Object tool converts a pan into an object, copying all of the 
information from the prototype into an object unassociated with that prototype. 
This permits the user to take a pan which almost meets his needs and edit that 
pan without aflTecting odier objects generated from the same prototype. 

Potymorphlsm 

Polymorphism is the ability of the system to automatically conven objects of one 
type into objects of another type so as to permit a tool which requires arguments 
of a cenain type to operate properly. If the object being convened to a different 
type is not chansed by the operation, then the polyma>rphism is temporary and 
transparent to the user. If the object being converted is changed by the operation. 
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then (he user is informed that the polymorphism will be permanent and given an 
opponunity to cancel the operauon. ' 

The pnmarN' form of polymoiphism is the reouired ability to conven ever>' object 
or pan in the system into a thansuiar mesh. Grouped objects can be con\'eraMi 
into a triansular mesh by taking the uiuon of the triangular meshes foimed by 
con%-erting their componenu into triangular meshes. Thus, the retpiiremem oFiiie 
system that it be possible to morph every object into a triangular mesh is 
essentially a requirement that every piece of primitive geometry in die svstem 
provide fiinciionaliix' for con\'ersioa to a triangular mesh. 

Gfouped objects which are permanendy moqihed into triangular meshes notlilili 
their hierarchical structure but as a collecnon of meshes instead of ofagects. 
Annotations, names, and other external database informadon is maintained mfliin 
tlw hieisrchy. 

PnfMb Primitives 

The system provides a ^et of geometric primitives which can be instantiated, 
moved. scaJed. displayed, and convened to meshes. These primitives are baste 
geometric shapes from which other geometry can be assembled such as the cube, 
cylinder, cone, prism, disc, and the like. Because no primitive specific operations 
are provided, adding a primitive to die system requires nothing more than 
providing a basic parameterizadon of the prinudve which captures its geomeir>'. 
a display generator, a conversion to a mesh, and an encapsulated daubase 
representation. The display generadon and mesh conversion routines taice a 
single floanng point value between zero and one as a complexity argument to 
assist in the detennination of the detail to provide ffor example.' the number of 
planar faces to use to approximate a cylinder). This argument may be set at any 
time by the user of die system. 

Prefab Primitives are inherently simple and are not used to represent objects such 
a& I-beams which can be consuucted as croups of other prefab primitives web as 
stretched cubes. 

PMrame^rized Prototypes 

It is a lon<(-ienn goal to provide genereU parameterized parts whose parameters 
can be set by the user. Vie concept behind scaling constraints ts a simplt 
version of the idea of getterally parameterizable prototypes. A parametenztd 
prototype would allow the user to set. for example, the width of a groove, the 
spacing cfa set of holes, or the number of legs on a stooL This would permit a 
smMer catalogue of prototypes to service a larger need but requires signiiicmly 
more complex funciumaiity for prototype description. 
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Structure Sliop 

The Structure Shop provides additional functionality for the generation of 
large, primarily planar structures such as buildings as well as support 
structures and basic functionality for point and line surveying. The StnictuK 
Shop adds the point, line, and plane objects to the palette available to the user 
and provides simple interactive functionality for manipulating these objecis. 

Extract Scan Points 

The user can cut out points from a range scan using the Extract Scan Points 
tool. The user selecu a range scan and then either traces out a loop around 
the points he wishes to extract or draw a polygon around these points. The 
range scan selected is replaced with two range scans containing separately 
the points inside and the points outside of the drawn region. Range scan 
names are augmented with a unique identifier (ie RS3S divides into RS3S.1 
and RS3S.2) and annotations are copied into each new component scan. 

Plane From Points 

The plane object a set of bounded polygons lying in a plane. The 
multi-<ontoured boundary consists of edges and venices which can be 
shared between juxtaposed planes as necessary. The Plane from Points tool 
calculates the bcst-ftt plane given a set of points. The set of points can 
originate either from a range scan or from a mesh (and by polymorphism this 
implies that thev can originate firom any geometric object or part). The 
boundar>' of the' plane is generated by projecting the points into the best fit 
plane and computing the convex hull. Depending upon a preference set by 
the user, the original object from which the points were dervied is deleted 
from the database or renuiins intact. The new plane object is inserted into the 
database. 

Corner from Points 

The Comer from Points tool generates three plane objects from a set of 
points. The set of points can originate either frem a range scan or from a 
mesh The three planes share a comer vertex and three edges extending from 
that vertex. With the exception of the shared vertex and a pair of shared 
edges, each plane object is bounded by the convex hull of its points when 
projected into its plane. Depending upon a preference set by the user, the 
original object from which the points were derived is deleted from the 
database or remains intact. The three new plane objects are inserted into the 
database. 

Add Hole/Add Region 

Holes and additional contours can be added to plane objects using this tool. 
In each case, the user draws a contour onto the infinite plane of which the 
plane object is a part. In the case of the hole, the resulung contour is 
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submcted from the given plane object causing the addition and deletion of 
veniees as necessar>'. In the Add Region case, the new contour region is 
umoned with the existing polygons and venices are added and remb\-ed as 
necessarx-. 

MenetivB Vert§xUovB 

The user can drag venices of a polygon with the mouse. These veniees aie 
constrained to lie in the plane spedfied bv their plane object. The user can « 
also identify one or two snapping planes to allow precise placement of 
venices. The addition of one snapping plane creates snapping regions 
around all of the venices of that plane as well as a weaker (smaller) snapping 
region ewry where in that plane. The user can easilv drag a venex so that it 
aligns with another plane or attaches to a vertex in anodier plane. If two 
snapping planes are indicated, dien all of the snap reeions described above 
are created for each of the two indicated planes: howewr. in addition, an 
extremely large snap region is created around the intersection point of the 
three planes (the two indicated planes plus the plane which contains the 
(lagged vertex I. If a venex is snapped to another venex bv the user, then 
that vertex becomes a shared venex. 

Vertices which are shared by two planes can be dragged only in the line 
which » the intersection of their two planes. Venices which are shared bv 
more dun two planes cannot be dragged. 

Venices also interact with the other countours within the same plane. 
Snapping occurs with all edges and venices of these other contours. When a 
contour which bounds a filled region is moved it mav intersea with other 
contours representing regions or holes. The result of moving a venex of sueh 
a contour is the union of all filled regions within that polygon. Similarlv. 
when moving a venex of a contour which bounds a hole, the filled regitin of 
this contour is subtracted from all other filled polygons in that plane. 

The new position of the venex is reHected in the database and edses which 
use this vertex as an endpoint are also updated. 

Unshan Vertex 

Qicking on the Unshare Venex tool and selecting a shared venex will cause 
shanng of this venex to be replaced with use of independent copies of this 
venex which may be moved independently and. in turn, separated from each 
omer. The original shared venex is deleted from the database and replaced 
with the copies. 

Vertex IttMrtion/Detetlon 

Venices may be insened or deleted with this tool. Veraccs are insened bv 
clicking on the location of the insenton point. Venices are deleted bv 
indtcaong the venex. Vertices which are shared are deleted from all 'of the 
plane objects in which they panicipate. All such chanees are reflected in the 
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database. 
Merge Planes 

The user can merge two plane objects with the Merge Planes tool. This 
operation checks that the two objects are near coplanar and warns the user 
before continuing if they are not. Merging planes causes the two indicated 
plane objects to be replaced in the database by a single plane object which is 
a best fit plane to all of the points from which the two original objects were 
generated. Vertices from the original plane objects are projected into the 
new plane while maintaining the constraints imposed by vertex sharing. The 
boundary of the new object is the union of the boundaries of all of the 
merged plane objects. 

Point to Fn9 Point 

The Point to Free Point tool generates a free standing point from either a scan 
point, a mesh point, or a vertex. The free point is a specially marked vertex 
which exists outside of any surface. Free points are a non-surface data 
object and thus cannot be converted into a mesh. Free standing points are 
primarily intended to permit basic surveying (propeny lines, etc) using the 
FDV. Free points can be named and annotated and arestored in the 
database. 

Make Free Une 

The Make Free Line tool generates a free standing line from two free 
standing points. The free line exists outside of any surface and therefore 
cannot be converted into a mesh. Free standing lines arepritiurily intended 
to permit basic sur\'eying (property lines, etc) using the FDV. Free lines can 
be named and annotated and are stored in the database. 
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Inquiry and Presentation 

The Inquiry and Presentation toolkit provides functionality for querying and 
displaying the data as well as for modifying non-geometric data stored within the 
database. This toolkit, while providing tight integration with the various Other 
toolkits in die system, will be separable into an independent sysam for 
distribotion to users who are not expected to use either the FDV unit or the model 
rectmstniction tools but who wish to access the information genmted by the 
various Cyra tools. - 

VI§wDatMWortd 

The geometric contents of a dau world may be viewed using an interaeiiM, 
vtsoal browser which displays a three-ditKensional repteseniacion of that dau 
world. Every geometric object or part belongs to one data world which is a 
subset of a database. Any number of dau world views may be opened and used 
independently. Operations which do not involve a particular viewpoint (unltke. 
for example, the Extract Scan Poims tool) may be per fo rme d using multiple 
views of the same data world by indicating schik operuids in one view "and other 
Is in any number of other views. 



Intmethfe Vhw Manipulation 

A data world view may be zoomed in and out. rotated, and panned by the user. 
These operations will be tnieracuve provided that the size and complexity of the 
model and the required frame rate do not exceed the ability of the piaif«B to 
generate frames at interactive speeds. : i- . 

MaiwetivaFly Through 

The user can navigate through a model using a fly through paradigm in npfdch the 
user controls the movement (speed, direction, and orientation) of an imajjj^rtary 
eye which views the scene. This fly through will be interactive provided that the 
size and complexity of the model and the required frame rate do not exceed the 
ability of the platfonn to generate frames at it 



Significant research Itas been done on optimizing this process to permit high 
frame rates on extremely complex models. It is our desire to ineoporate such 
nsearchinlatergeiurationsafOuCGP. 

SmwfLoadNamadVlaw 

The user may save a current viewpoint in a data world with a name and 
annotation. A named view may be loaded into any daa world viewing window 
thereby replacing dut window's viewpoint widi the saved viewpomt. 

Delete Named Vlaw 
A named view may be deleted from die 



database at dw request of die um^. 
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Standard Views 

The six standard views (left, right, front, back. top. and bottom) relative to a 
user-specified origin and at a user-specified distance arc immediately available 
in any data world view. 

Orthographic/Perspective Toggle 

A data worid may be viewed using either an orthographic or perspective 
projection. Operations from all of the various tooUdts will fimciion conectly 
regardless of which projection is being used. 

IntegFatB Osfs Worids 

The user can merge one data world into a second data woiid by specifying pairs 
of corresponding paints in the two data worids. The user must provide at least 
three non-degenerate pairs of points. A transformation which repositions the 
second dau world so that the correspondences will be met as closely as possible 
(in a least squares sense) is calculated from the point pairs. The second oaia 
world is removed from the database and all data within that data world is copied 
into the first data world with the calculated transfomaaon. 

Distance 

The distance between two user-specified locations in the model can be 
calculated and di^ayed using the Distance tool. 



remaindar of this aaeUon la vagua and Incomplete. I need to finish 
working this out but It atiould and up bring a very general way of 
buying much ground toward 'Intallligant'' modalllng) 

Viewing Parameters 

Every object, pan, or primiiive can provide any number of named visual 
representations. The user can specify which of these named visual 
representations is to be used for a given object, pan, or primitive or for an entire 
scene by setting an object's "representation attribute". £veiy object is provided 
with a jefiiult representation which is used if the named viiuil repicsemoiion 
does not exist for that object or if no named visual re pr esen tation is provided. A 
named visual represenution is either a system provided drawing s^le such as 
wireframe or bounding box or else a pmnier to another model such as a lower 
resolution version of the model or a symbolic reptesentition of that object (two 
cones representing a complex valve). 

Typically, a primitive provides at least the following representation attribute 
values of which "surface" is normally marked as the default: wirefnune, hidden 
line, surface, bounding box. and invisible. 
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SurRice property paranwiers such as color and reflectivity are also provided 
dupugh die visual representation by providing attxibute/value pain as oan of an 
object s visual representation. An entity in a hierarchy uses its doses ance«nr«- 
vjJue for an attribute if no value is specified for that attribute in the entity. EvSv 
attribute requir^ for the drawing of an object (such as color or fine width) must 
be provided with a conesponding default in case no value is specified bTie 
enmy or its anceston. TTic aoributes which will be supported^all piSii™ies 
in the first release of die CGP are ambient color, diffuierolor. specX color 
emissive color, shimness. transparency, show name, and show annotiS In 
addtuon certain pnmitives wilf support the "show dimension" attritefeTianK 
scans will support die "use intensity pseudocoloring attribute". ^ 

Cnatenjiad/Save/Del§tB Named Visual RapnsantaUon 

a named visual representation to pennit him to vjew a database 
larameter set dian odiers using die database. This viiial 
n is given a name and is stored in die rfat^ bnf When a «— — rtwnt^c 
a nrar visual representation, all of die current attribute/value pain fiS, obiect 
inthe ditt worfd is associated with the named visual representation oSges 

'**^*^? ^'"i^^ " representatio^Sbute 
of '" K> wtrefiame or its color to blue) when using diis named visual 
re{msentatton ^ct only die attribute/value pain of die currendy loaded visual 
representation. TTius. a piping engineer might generate a visual representation of 
a dau world in which all pipes are displayed as cylinden during oTe snsion^ 
a second visual representations in which pipes are displayed as center lines to be 
used dunng another session. This, of coune. presumes that die pioe obieet 
provides a named repiesemation attribute for center lines. »^ 

^^^lE?!?" PMrfip permits many simultaneous views on die same database 
vfewer"* wmout requinng separate copies of the database for each 

Cnata/Delete/Wew/Edlt Annotation 

An annottdon is a typed data field associated widi an object, primitive Dart or 
prototype. Annotations pemut die inclusion of application specific dta in the 
CGPdatabase. Thisdaucanbeinany ofdielStfstandardS ypS^^ 
recall what diese are but diere in only half a dozen of diem and itSym 
supported by every database). Each annoution is referenced by name 
Armounons assodated widi a prototype apply to every pan genenued'wtdi diat 
prototype unless die value of die annotation is cxplicidy oveF-ricSSTlJTtlw^^ 

Eveiy entity has. at a minimum, a name and type annotation. 

SelactbyQtmy 

The user can select a set of entities using an SQL query on die amwitarifHw «n 
entities in . dau worid. Surt queries aAh«,aSSir.o^e^SSS1.i SI 
degree of SQL suppon provided will be based on die database seSSthe 
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entiiies which are retunied by the queries are marked as selected and may be used 
as arguments to all ttwls which permit operations on multiple objects. D^ta 
stored by the CGP widiin the database is also accessible to the user via a 
read-only front end to CGP's database stiuciures. 
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Data Exchange 

The Data Exchange toolkit provides tlw functionality necessary to seamlessly 
integrate the CGP with other CAD pacicages and vendor provided catalogues. 
The toolkit will ultinutely impon and export a wide varieqr of file types 
dependent upon market need. 

ImporMExport CRX "^''^ 

The export function generates a single binary dau file represendng selecied 
objects, a data worid. or a database in CRX. Cyra's proprietary dan fonmt. The 
inqmfuiictionpeniiits the reading of this data fiHinat. 

Import/Export DXF T 

The export function generates a DXF data file representing selected ob^ets, a 
dau worid. ma database in this general purpose CAD standard formtt. the 
DXF repiesentaiion is limited in the following ways: 

Import DXF file loads an entity or sm of entities stored in a DXF file ieifrlhe 
CGP. The CGP version of diis DXF file will be limited in die following ways: 

hnporVExpert RGB 

The export RGB function generates an RGB video in»ge file from a seta^ 
video image. Impon RGB generates a video image object in die CGP tsdm an 
RGB video image fik. 

Import^port tnvBntor 

The export Inventor function generates an Inventor file representing selected 
objects, a data world, or a database in die Inventor file format. The resulting 
model when viewed using a standard Inventor viewer will be identical to the 
model as viewed in the CGP. The inqiort function loads an entity or set of 
entities stored in an Inventor file into the CGP. The CGP version of this Inventor 
file will be limited in the following ways. 

Export PostScript 

The export PostScript function generates a PostScript file or prints direcdy to a 
printer the view represented in a selected dau world. 
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General Interaction and System Information 



Interaction with the toolkits will be through a mouse and keyboard widi emphasis 
on visual interactivity. The user interface is expected to have iconic tool palettes, 
function key shortcuts, menus, and windows. Most of the fixnctionality will be 
accessible using only the mouse. The user interface of die system will be 
consistent with standard interfiles and interaction paradigms on a given 
plaifmn. 

Plttfoiw 

The CGP will run on a 100 MHz Pentium Plooessor miming Windows NT in a 
laptop box. The system will provide a minimum of 48 Kfbyies of RAM widi a 
5 1 2 kbyte cache The platform is expected to have an 800x600. 256 color display 
and at least one Gbyte of disk space. The system will communicaie widi the 
hardware via an ethemet over PCMCIA. 

The first port of the CGP is expeaed to beta a Silicon Graphics ^atform to 
provide in-the-qffice model reeonstnietion in a graphics intensive emnronment. 

tMpSystmn 

The CGP will provide an interactive help system which provides context 
sensitive help in a format coiresponding to die standard interface for applications 
on a given platfonn. All functionality accesuble to die user will be documented 
in die help system. 

Undo 

The CGP will allow dw user to Undo any number of openuons (to die fixnctional 
limits of die system such as memory and disk space) perfbimed during a session. 

DoGunwntaUon 

Hard copy documentation will be provided for die system inducting, at a 
minimum, a field guide which can be used by a surveyor on-site. 
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Cyrax 
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Overview 



This CONFIDENTIAL overview of Cyra Technologies 
(Cyra) and its unuiue. laser-based daca acquisition, 
recogniiioa, and model construction technology has been 
prepared to aid potential strategic partners, investors, and 
collaborators in bener undersauding and evaluating Cyra 'S 
potential. 

Cyra Technologies was established as a California 
Corporation in 1993 to research, develop, and market 
innovative high-technology computer-based products 
the AEC and manufacturing indusoies. 

Cyra is currendy developing a line of laser-based spatial 
imaging and synthedc vision systems for three-dimensional 
data acquisition, recognidon. and model construction ma 
broad spectrum of markets. The initial product. "Cynu". is 
a ponable laser-radar scaimer plus graphic computer capable 
or seini-automatically cieadng ifaiee-dunensional (3D) 
computer models of anytbtng in its field of view. A hybrid 
cameia/surv'eying insituinem. tiean generate 3D computer- 
aided design (CAD) models of everything from hillsides to 
buildings to aiictafi. With millimeter accuracy at a range of 
over 50 meters. Cyrax is targeted at the 
.Architecmre.'Engineering/Consirucdon fAEQ industry, as 
well as the mass market 3D spatial data demands in the 
manufacwring. industrial, entenaiament. and multimedia 
areas. 



The Need for Spatial Information 

Data AcBuinittfw Accurate information about our surroundings is crucial to 

modem life. . From the video game designer modeling a 
dinosaur character. O) geologists mapping a strip mine, 
construction firms readying "as-built" plans of a skyscraper .- 
or the aircraft designer who requires precise three- 
dimensional measurements of complex airframe pans, 
modem society has a constant need for accurate d ira on our 
spatial surroundings. 

The more than SI trillion expended every year by the 
worldwide construcdon. utility, transportation, and 
manufacuring industries relies in large pan upon fast and 
accurate spatial data— with these industries spending at 
minimum one tenth of one percent of their revenue on data 
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acquisition, the need exceeds 57 biU^A ( US) annueUly'. 
Domestic sales of surveying total stations alone today exceed 
S200 million annually. 

Data-Based Positioning In addition to our need to know the locations of objects in 

our suiToundings. there is enormous demand for the accurate 
'positioning' or placement of objects. When a fence is 
erected, a building foundation poured, a freeway built, or a 
complex machine assembled, the building f^cator needs to 
be able to precisely posidon the pans. With "Quality is Job 
1 " a national slogan and with extreme pressure on corporau 
margins, manual methods are no longer acceptable. 

.Although currently largely unaddressed by vendors, the 
market for data-based posidoniag is rapidly growing. As 
our society moves to ever more flexible manufacturing and 
customizadon of consunmble goods, the demand for data- 
based positioning systems will explode. 



Traditional Methods 



Acquisition-PninK &. Lines Traditionally , society has relied upon cumbersome manual 
techniques for data acquisition. In the AEC arena, 
surveyors with optical transits, levels, steel tape, and plumb 
lines spend long hours in the field coUecting data, while in 
manufacturing, machinists and designers with calipers and 
precision machine tools are the primary gatherers of 
infomuiuon. 

These tools and methods have been refined to a high level 
of precision, and in recent years even partially automated. 
However, they rely upon the slow, laborious, and enor- 
prone gathering of twoKlimensional data. From the points 
and lines measured in the field or ac the workbench, data is 
taken back to the drafcit^ table or CAD system and 
tediously added together to build a mpographic map. site 
drawing, or a three-dimensitmai nuxiel of a part. In 
addition to the time involved, the current process suffers 
from human error. 

Posiiioning .Accurate positioning has always been a challenge. Most 

civil and constrwtion works are 'staked out' point-by-point 
by survey teams using total stations (the modem surveyors 
tool— an integrated teiescope/angle/distance meter) and 
other surveying equipment. Although there have been some 
advances (new total stations can help guide one to the exact 
point with visible light beams for example), the process is 
still slow, laborious, and pfoac to operator error. 

In manufacturing similarly, widi products becoming 
increasingly complex and manufacturing requirements ever 
more exact, accurate pan positioning is a constant, and 
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growing challenge, Numerous vcR r— » — 

develop machine vision devices, bat most such systcmslhave 
mndamemal limitations and are designed for single 
applications. On assembly lines, robots can often be 
programmed to position objects— as long as they are wiihm 
reach! However, with big assemblies such as airframes and 
ship hulls, rabficators have had to resort to less than optimal 
solutions such as cumbersome mechanical 3D arms 
(typically used for scanning) or surveying equipment to pre- 
marts dnU holes or alignment guides before manually 
and assembly. ' 



'Spatial' Opportunities 



The explosion of computer-aided design (CAD) in the last 
decade, as witmssed in the phenomenal market success of 
Autodesk Gnponuion and others, has accentuated the nf fd 
for ftst. accurate. 3D data, in machine-ready form. Widi 
almost every engineer and designer having moved from the 
pomts and Imes of die T-square and pencil to the glowing 
phc«rfiors lUufflinating surfaces on the computer screen, the 
need for spanaldaa has skyrocketed. Tliecunent process 
of manuaUy surveying or measuring hundreds of points. lo 




AiHoCAD 



.Automated PfwiriQniny 



w. «>tT«Tuig mcasunng nunoreos or points. £0 

betaer assembled via software into lines and eventuajlv® 
representations for use by engineers and designers in C^Wfe 
programs, is inefBdent and financially burdening. 

A system to gadier entixe 3D objects in the field as complete 
Sttinces woukl eliminate several steps in the current data 
aeqoi^ion process in addition to improving accuracy and 
provubigda^ 



In addition to mainstream acquisition opporranides. theie 
are numerous growrh possibilities in acquisition as well. 
Among the many mentioned in market study intervnsws. 
some of the mote interesting include remote 3D 
reconnaissance/vision, machine vision/recognition, aeiiai- 
based acquisition (as a replacement for aerial 
photognunmeity), and underwater vision. 

In the fomre, as fuU duee-dimensional design along the 
luies of diat being introduced at firms such as Boetne 
becomesmore perv-asive. the abiUcy to pteciselv position 
objects ftom buildmgs to automotive parts, directly from the 
designer $ computer screen, will become the nonn. Cyta is 
workuig on a relatively simple and low cost extension to Us 
acquisition technology to addiess the spatial positionini 
problem. ■ 
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A. Introduction 



Challenged by the need for accurate spatial information in 
the AEC industry and Ae absence of tools to provide it. 
Cyra Technologies is developing a portable, three- 
dimensional data acquisition tool. Cyrax. Similar in many 
respects to a camera, the instrument under development 
adds the capability to capmre the three-dimensional 
coordinates to the world of "points" making up the field of 
view in front of it. Cyrax combines the camera's speed and 




high volume arauisition with the capture of 3D spatially- 
accurate dau. liie Cyrcx design specificauons call for a 
precision, hi^ speed digital instrument ^obk of in-field 

miUimeter and another aceunue to 6 millimeters) at distances 
of up to SO meters. Cyrax' s software, which runs on a 
small portable computer, couples with the hantwaxe to 
provide sophisticated point processing and a friendly 
inter&ce to the hardware. 

Starring widi tl^ design requirements for a useful device for 
the AEC professional, an assessment of potendal 
technologies was made. Although the preliminary 
specifications for Cyrax were beyond the scope of any 
existing data acquisiuon methods. Cyn pushed ahead 
focusing on laser technology, and laser radar in particular. 
An extensive search of university and govetnroent 
laboratories, as well as intensive research by the Cyra team, 
led to the creation and assembly of die core technologic 
components necessary* to meet the design specifications. 

Using laser radar and extremely high-speed pico-second 
electronics. Cvrax was designed to capture thousands of 
diree-dimensional coordinates per second in broad daylight. 

The Cyrax software is as sophisdcated as the hardware. 
The innovauve. user-friendly code allows die operaror to 
direct the scanning process and to generate meaningful data 
from the resulting point cloud. The software team is 
currendy pushing die limits of automated object recognition 
and generation, having developed semi-automated 3D object 
segmentation routines as well as f»si and easy surfece and 
object creauon tools. 
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B. Laser Radar 



Despite its discovery more than 30 yean ago. 'lidar' or 
laser radar has seen limited application outside the defense 
industry. The dynamics have recently changed with tte 
development of simpler, lower cose micrtKhip lasers and 
with the push to open govenunent defense researdi to 
industry. In 1986. with the first annual Laser Radar 
Conference, lidar technology truly came into its own. 

Lidar. like tradidonal radar, is based upon the precise 
measurement of the time of return of a light pulse or the 
precise measurement of phase shifts of light pulses. The 
use of the ughdy constrained and 'aimable' beam of 
coherent li^ waves from a laser offers distinct advanta|;es 
in accurate targeting. Until recendy, however, laser radar 
was technoiogicaUy constrained. Fust, few lasers safely 
offered sharp enough energy pulses for their emission to be 
measured widiout placing a reflector on the target. Second, 
high speed precision eieeixonics such a time of flight 
measuiemems of focused. Ugh power laser emissions were 
liniied in accuracy doe to die lack of electronics with the 
pico-second response times necessitated by a 'speed of 
light' measurement system. As a result, systems such as 
laser ranging total stations have had to take hundreds or 
even diousands of nuasuremems to avenge out errors of 
die dming eltxironics . 




The recent development at a leading national laboratory ttf a 
microchip laser capable of extremely sharp, high energy 
pulses has opened die door to new possibilities for laser 
radar systems. Moving quickly, Cyra Technologies 
obtained the exclusive worldwide lights to this patented laser 
in Cyra's markets. 
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C. The Cvrax Svsxem 



Cyrax is composed of 3 priinvy components: 
■ (he electronic laser scamet. or 

"FiaU Digital Vinon" imdaiie 

• the FDV conirei. urfeting and image 'assembly' software, or 

"Compuur Craphie Ftreaption " module 

• the communicatiom software to existing 3td party CAO 

Devg^nnmfnr .^lralg9^^ Cvrax is being developed by a team of iueniational expeits 
under the direcdon of Dr. Jetty Dimsdale, Cyia's Vice 
President of Technology. The MIT Lincoln Labotaioty, 
under funding by Cyra, developed die lidar transceiver 
(transmitter and receiver) pan of the FDV. The balance of 
the FDV. including the scanning system and electronics, 
were developed by Cyra and several consultants and 
vendors. The Compater Graphic Software development is 
being done at Cyia widi die aid of researchers from die 
University of California at Berkeley Ail of die development 
is proceeding in close collaboraiioii widi Cyta's markeung 
team and with key indtuiry playen who have been 
confidentially briefed and bnught in dming the development 
cycle to provide input and help guide die evoludon of die 
technology. 



FDV The portable Field Digital Vision system has 3 main 

elements: 
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CGP The CGP is the 'bndns' of the system. A complex amy of 

state-of-the-ait graphic software, it pushes as many 
technologic limits as the laser scanner itself. Providing an 
inntitive gr^hic interface, it allows the user to target the 
areas and levels of detail to be scanned on a "video" view of 
the world in front of it. As the FDV scans the selected " 
portions of a scene, the CGP displays the pohits in a. seeisd 
depth-shaded thxee-dimensiMial view. 

It is in this second window of points that the power of die 
CGP becomes clear. Algoridims for building up objects 
enable the user, with a few mouse clicks, to mm the 'cloud 
of points ' into CAO-teadable planes, cylinders, and 
surfaces. Understanding the need for accuracy, the software 
retains all scanned data points throughout the process (thus 
allowing iu optimal fit alsohihms to utilize any additional 
data diat nuy be added later). Unlike traditional surveying 
and scanning equipment, all of this data assembly can be 
done extremely rapidly and in the field. This offers clear 
advantages— data can be easily spot checked on site to 
eliminate obvious (but all coo frequent in traditional practice!) 
mistakes. As the model is assembled, areas where more 
detail is needed become apparent and more data can be easily 
added. 

For situations when an entire objea cannot be captured from 
one location. Cyra has designed software to "zip" sets 
scans togedier imo one larger piece. 

The CGP is based on open architecmre and is written for 
portability and ease of modification in the future. The 
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CAD Link 




development has taken piace on S ilik^n Graphics 
workstations using the object-oriented ANSI C-w- language, 
coupled with the industry-standard OpenGL and other 
industry standard graphic environments. Since Cyrax is a 
poitable. field-operable system, Cyta has also adapted the 
COP to run on generic lightweight Intel-based laptops. Cyra 
has successfiiUy demonstrated the CGP running on an Intel 
486 laptop under Microsoft WindowsNT. 

Cotient plans call for several computer hardware options to 
be offered with the system (see Appendix L for a 
preliminary pricing plan). These range from a hybrid pen- 
based IBM Pentium ThinkPad to a mote oaditional Pentium 
or PowerPC laptop with expansion sloa for one of die 
many gcaphie accelemor boaids (such as the OKI America 
Triingl OpenGL acceieiatar for WindowsNT). For users 
who demand die ultimate and have access to a fixed power 
source, an SGI workstation can be offered. 

Rather than reinvent the wheel. Cyrax will tie into existing 
CAD software for its traditional modeling, editing, and 
other data manipulation needs. By allowing the purchaser 
to continue with their own CAD system (most Cyrax users 
will already have some form of CAD package), die software 
and support costs are kept down and compatibility is 
assured. Tlinugh support for the industry-standard 3D 
DXF file format, die <jGP can be linked to ahnost all 
existing CAD programs. Support for odier 3D data formats 
is additionally being considered for specialized sectors 
(such as the autcmiotive market) where offering enhanced 
features might prove beneficial. For those that may want 
tighter integratuin with a specific package, the open 
architecture of die CGP allows relatively easily coupling 
with programmable CAD systems— a demonstration of the 
CGP tunning within the Iniergraph's RoadWorics software 
was conducted during the summer of 1994. 
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D. Overview of Competitors' Technologies 

De^te the differences between the 3 target market 
segments, the underlying technologies axe broadly shared. 
There are two basic cacegories of 3D acquisition 
technology —concBcr (such as probe or tape, etc.) and nan* 
cofiiacr(suchasalaserandphotognunmetry). Contact 
systems involve bringing a movable measuring device 
(typically a needle-pointed wand or a mixxored leflector cm a 
stake) to the point to be dimensioned. Non-contacc 
systems, on the odier hand, lenintelv measure (often wkh 
light reflection) the spatial locations of points on an object 
physically separated £tom them. 
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Each technology has advantages and disadvanuges. Piobes 
tend to be more accurate, but are typically slow and 
cumbersome (since they gather data point by point). Non- 
contact methods tend to be faster (as they eliminate die need 
for traveling to or coming into contaa with each point to be 
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dimensioned). As seen in the table awive (ind illustrated by 
the diversity of systems utilized), before Cyn's 
breakthrough, no technology held a clear performance edge 
over any of the others. 

An exhaustive worldwide search has found over 70 vendors 
selling various forms of 3D data capmre systems, from 
mechanical arms, to GPS receivers and laser scanners. With 
the excepdon of'tonl Italian' and 'GPS total station' 
technologies, the vast mqocity of diese are small, under- 
capitalized and untter-commiiiBd firms selling expensive, 
Itminsdfuncdonality systems in small volumes. Full product 
and vendor descriptions, as wdl as photographs, and firm 
by firm compeddve reviews are avaUabie in Appendix B. 

Of die many competitors widi datt collection systems, only 
two can gadier 3D dan at large (SO to 100 meters) distances: 
MENSI and SMX. Altfnagfa diese laser-based systetns may 
appear similar to Cyra, each has severe limitations. The 
SMX Chesapeake laser imetferometry system is priced over 
SlOOjOOO and requires a probe to be placed at each point to 
be dimensioned, limiting its ability to capture 3D stvfaces. 
Its advantage is high accuracy. The MENSI Soisic uses 
laser trianguladon but is liniied to gathering 100 points per 
second, is not extremely portable (large, heavy and requites 
AC power), and costs close to S260/X30. Neither system is 
believed to have the software sophistication of Cyrox. 
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1. An integrated system for generating a model of a three- 
dimoisional scene, comprising: 

a scanning laser device that scans the three dimensional scene and that 
generates a point cloud, the point cloud comprising a plurality of data points 
that each represents a location of a corresponding point on a surface of the 
seme; 

means responsive to the point cloud that generates a first naodel 
rqnresenting constituent gecnnetric Aapes of the scene. 

2. The system of claim 1, and further including: 

means responsive to the first model for generating a data file, lA^ieitin 
the data file is suitable for input to a computer-aided design (CAD) sysfqn. 

3. The system of claim 2, and further including: 

means to initiate execution of the CAD system and automatically 1 
loading the data file into the CAD system. 

4. A method of onitrolling the timing of output pulses from a laser 
for use in a device which requires scanning of the laser output, whoein eidi 
ouQmt pulse is generated in refuse to a pump pulse comprising the ttqit 
of: 

monitoriiig the time delay between the initiation of the pun^p 
pulses and the subsequent generation of the associated output pulses; 

predicting the time delay between the initiation of next pimp 
pulse and the associated output pulse based on the monitored time 
deli^s and; 

initiating the next pump pulse at a time selected to inaire the 
ou^ut pulse is generated at a time to permit proper positioning of the 
laser ou^nit during the scan of die beam. 
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5. A method of manually sqiarating from a plurality of clouds of 
points, representing three-dimensional features in a scene, a subset of the 
points that represents a desired feature in the scene, the method comprising: 

selecting all the point clouds that include at least some data points 
5 representing the desired feature; and 

changing a view of the clouds and drawing a polygonal lasso to refine 
a selected subset of points to be included in a point sub-cloud and repeating 
the refming as many times as required to obtain the desired sub-cloud. 
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6. A method for automatically segmenting a scan field of a scene into 
subsets of points that represent different surfaces in the scene, comprising the 
steps of: 

separating the scan field into a depth grid that includes depth 
information for scanned points of surfaces in the scene and a normal grid that 
includes an estimate of a normal to scanned points of the surfaces; 

convolving the depdi information of the depth grid to generate a depth 
rating image Mibosc values represent a gradient of depth diange fhim one 
scanned point to another scanned point in the scene; 

convolving die cmnponents of the normal grid to generate a scalar 
value for each componoit for each point of the normal grid, 

for each point of the normal grid, determining firom the scalar «alii» 
for the components of that particular point a gradient of the normal dMif ' 
point, ^dierein the gradients determined for the points of tiie normal gM 
collectively constitute a normal rating image; 

converting the depth rating image to a binary depth image using a 
recursive thresholding technique; 

converting the normal rating image to a binary normal image usbg ft 
recursive thresholding technique; 

combining the binary dq>th inuige and the binary normal image to 
detennine a single edge image; and 

grou|Mng subsets of non-edge points as belonging to conespondiiv 
surftoes of the scene. 

7. A method as recited in claim 6 further including the steps of: 
determining the type of geometric primitive that would best first eadi 

groiq> of points; and 

fitting the geometric primitive to the data points. 



8. A metiiod as recited in claim 7 fiirtiier including die stq> of 
intersecting adjacent planar regions in the scene. 
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9. A method for fitting a point cloud representing a comer, 
comprising: 

determining a fit of three planes to the points of the point cloud and 
creating the planes for a model; 

determining the three lines at the intersection of pairs of planes and 
creating the lines for the model; and 

determining the vertex point at the intersection of the three planes and 
creating a vertex point for the model. 

10. A method for modeling a three-dimensional scene, comprising: 
generating a plurality of points that each represent a point on a surface 

of the scene; 

determining a best fit of a cylinder for a group of the points using 
surfoce normal estimates and global error minimization. 

11. A method for modeling a three-dimensional scene, comprising: 
generating a plurality of points that each represent a point on a surface 

of the scene; 

determining a best fit of a cylinder for a group of the points using a 
quadric surface fit and global error minimization. 

12. A method for modeling a three-dimensional scoie, comprising: 
generating a plurality of points that each represent a point on a surface 

of the scene; 

determining a best fit of a sphsK for a group of the points using a 
quadric surface fit and global «ror minimization. 

13. A method for modeling a three-dimensional scoie, comprising: 
generating a plurality of points that each represent a point on a surface 

of the scene; 

determining a best fit quadric surface for a group of points; and 
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detemiming wfaidi geometric primitive of a plurality of the fiunOy 
described by the quadric surface best fits the group of points. 

14. A method for merging two geometric primitives of the same type 
to form a single geometric primitive of the type, comprising: 

creating a new group of points by combining the points used to 
originally fit each of the two primitives; and 

fitting the new geometric primitive using any iq>|»opriate fitting 
technique and the newly geneialed point group with points from each of die 
original primitives. 

15. A method of registering a first model, consisting of a plurality of 
points and geometric primitives and having a first coordinate system, wifli a 
second model, consisting of a plurality of points and geometric primitives Ind 
having a second coordinate system, comprising: 

identifying by a user common features of the first and second 
scenes ; 

identifying a transformation between coordinate systems that is 
responsive to the identification; and 

transforming the objects of the second model so that they use 
the first coordinate system. 

16. A method of warping, comprising: 

selecting one or more models represented by a plurality of 
point clouds and geometric primitives; 

sptdfying constraints on the locations of any number of points 
or getHoetric primitives: 

creating an artificial volume that surrounds the points and 
geometric primitives in each view and assigning mechanical material 
duuacteristics to the surrounding volume; 
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computing a minimum energy configuration for the material in 
the surrounding volume in which the points or geometric primitives 
are embedded such that the configuration satisfies all applied 
constraints; and 

displacing the points and primitives in accordance with the 
computed minimum enorgy configuration of the surrounding volume 
of material. 

17. The method of claun 16. wherein the constraints are specified to 
eliminate closure errors. 

18. An integrated system for generating a model of a three- 
dimensional scene, comprising: 

a scanning laser device that scans the three dimensional scene with 
pulsed laser beam wherein the pulses of light last less than 1 nanosecond with 
up to .2 in each pulse and measures the time delay, with a resolutimi of 30 
psec or less, between each emitted pulse and a corresponding pulse that is 
returned from a surface of the scene and wherein said scanning laser device 
further tracks and measures the angular orientation of the beam during the 
scan; and 

means for generating a point cloud based upon the measured time 
delays and angle measurements, the point cloud comprising a plurality of data 
points that each represents a location of a corresponding pomt on the surface. 



-129- 



19. A system for calibrating the measuring electronics in a device 
which reqiures monitoring the time of flight of the o\xtp\A pulses fiom a laser 
comprising: 

a single mode optical fiber iivith one end thereof positioned to 
receive the output pulses of the laser, said single mode optical liber 
having a known length; 

a detector positioned at one of the ends of the fiber for 
monitoring when the pulses exit the fiber and generating a signal m 
response thereto, said signal being passed through the measiirmg'^7 
electronics; and 

a processor for calculating a theoretical length of the fib^^ 
based on the detection of the pulse exiting the fiber and comparing 
that calculated length with the known length of the fiber to calibnto 
die measuring electronics. 

20. A system as recited in chum 19 wfaeiein said optical fiber 
includes partial reflectors located at each end diereof so that for each laser 
pulse entering the fiber a train of pulses will exit the fiber and wherein mA 
train of pulses are used to fiuther calibrate the measuring electronics. 

21. A system as recited in claim 20 further including delay 
measurement electronics and wherein die train of pulses have a fixed deliif 
dierebetween indwreby the monitoring of the tram of pulses can be used to 
calibrate the deby electronics. 

22. A system as recited in claim 21 fiirther including a means for 
varying the power of the pulses monitored by die detector and wfaeiein md 
detector fimctions to generate a signal when the power of the detected light 
exceeds a predetermined threshold and wherein said processor fimctions to 
track the variation in the deky of the generation of the output signal by the 
detector as a fimction of the power of the output pulses, said processor 
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fuither functioning to calibrate the measurement of the delay based on the 
measured power of successive pulses used for monitoring the time of flight. 

23. An apparatus for obtaining position information about surface 
points of a three dimensional object comprising: 

a laser for generating an output beam; 

scaiming system for moving the laser beam over the object; 

monitoring system for automatically measuring the range to the object 
based on the measurement of the reflection of the laser beam, said monitor 
system also tracking and measuring the angular position of the laser beam, 
said monitoring system having a positional accuracy for each point in three 
dimensional space equal to or better than six millimeters at one standard 
deviation over a range of up to 100 meters. 

24. A system as recited in claim 23 wherein each range 
measurement can be made in under O.OOS seconds. 

25. A system as recited in claim 23 wherein the laser generates a 
pulsed output and the energy per pulse is less than 0.2 micro joules and the 
average output power of the laser is no more than 1.0 milliwatts. 

26. An apparatus for measuring the distance to an object 
comprising: 

a laser for generating a beam of output pulses; 

a monitoring system for measuring the distance to the object based on 
the reflection of the laser beam, said monitoring system having an accuracy 
equal to or better than 6 millimeters at one standard deviation over its entire 
range of iqi to 100 meters and wherein each measurement can be made in 
less than O.OOS seconds and wiicrein the laser has an energy per pulse of no 
more than 0.2 micro joules and an average power of no more than 1 
milliwatt. 
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27. A system as recited in claim 26 where the object is provided 
with retro reflectors and where the range of operation is up to one mile. 

28. An apparatus for acquiring three dimensional information firom 
a remote object comprising: 

a scanning laser module for measuring position information of the 

object; 

a video module for caphiring image information from the object; and 
a processor for rendering a model of the object w^ch include 
position infonnation and the image information. 

29. An apparatus as recited in claim 28 wherein the video im^ 
information is collected in a spatially coincident manner with the 
measurement of position infimnation. 

30. An approach as recited in claim 29 wherein the video image 
infimnation is collected from points adjacent to the points where positim 
information is obtained. 

31. An apparatus for obtaining positional information about sur£K» 
points of a three dimensional object comprising: 

a scanning module for measuring three dimensional position 
information about an object; 

a video module for capturing and di^laying image infcxniatkm 
from the object; and 

a processor operating with the scanning and video modules and 
permitting the use of the image information cultured by said video 
module to aid in targeting the scanning module. 

32. An apparatus as recited in claim 31 wherein said processor 
functions to specify a portion of the object to be targeted by the scanning 
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module by dragging the image of an outline over the video image of the area 
to be targeted. 

33. An apparatus for obtaining positional information about surface 
points of a three dimensional object comprising: 

a scanning module for measuring three dimensional position 
information about an object; 

a video module for displaying image information obtained 
from the scanning module; 

a processor operating with the scanning and video modules and 
permitting the use of the image information displayed by said video 
module to further refine the targeting of the scanning module. 

34. An apparatus for obtaining positimud information about surfisice 
points of a three dimensional object comprising: 

a scanning module for measuring tfiree dimensional position 
information about an object, said scanning module including a laser 
for emitting a beam of visible radiation; and 

a processor for controlling the scanning module and wherein 
said laser beam can be manually positioned so that the visible beam 
will target the portion of the object to be scanned in re^nse to a 
control signal from the processor. 

35. A system for calibrating the measuring electronics in a device 
which requires monitoring frequency changes in a light beam generated by a 
laser used to measure distance to an object, wdierein said beam has frequency 
chirp imposed thereon comprising: 

a single mode optical fiber with one end thereof positioned to 
receive light from the laser; 

a detector positioned to receive light which has travelled 
through and exited the fiber in combination which light from the laser 
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which has not traveled through the fiber, said detector for motiitoring 
the changes in the ficqucncy of the combined beam; and ""^ 

I»occssor for determining the linearity of the chiip on the 
beam based on uniformity of the fivquency clianges measured by the 
detector and using the result to calibrate the measuring electronier 

36. A system as recited in claim 35 wherein the fiber has kaam 
length and includes a partial reflector on said one end and at least a fwtiai 
reflector on the other end, and wherein light reflected from said one end of 
the fiber which has not traveled in the fiber is measured by the detecttit& 



11 the processor further functions to calculate a theoretical lengHi^tlie 
fiber based on the frequency changes measured by the detector and c 

that calculated lengdi with the known length of the fiber to calibrate the 
measuring electronics. 
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